* Earlier initialization of elf info.
Everything used for elf info initialization comes from the param.sfo, so we can initialize this earlier to have this information accessible during memory init.
* Extract compiled SDK version from pubtoolinfo string
Up until now, we've been using the game's reported "firmware version" as our compiled SDK version. This behavior is inaccurate, and is something that has come up in my hardware tests before.
For the actual compiled SDK version, we should use the SDK version in the PUBTOOLINFO string of the param.sfo, only falling back on the firmware version when that the sdk_ver component isn't present.
* Store compiled SDK version in ElfInfo
* Limit address space for compiled SDK version at or above FW 3
Sony placed a hard cap at 0xfc00000000, with a slight extension for stack mappings. For now, though stack mappings aren't implemented, there's no harm in keeping a slightly extended address space (since this cap is lower than our old user max).
Limiting the max through address space is necessary for Windows due to performance issues, in the future I plan to properly implement checks in memory manager code to properly handle this behavior for all platforms.
* Use compiled SDK version for sceKernelGetCompiledSdkVersion
I think this is pretty self explanatory.
* Log SDK version
Since this value is what most internal firmware version checks are against, logging the value will help with debugging.
* Update address_space.cpp
* Update emulator.cpp
* Backwards compatible logging
Because that's apparently an issue now
* Define latest released firmware version, use that for sceKernelGetSystemSwVersion
I feel this is less hacky and error-prone than just returning the game firmware.
* sceKernelGetAllowedSdkVersionOnSystem
* sceKernelHasNeoMode
* sceKernelGetAppInfo stub
* sceKernelGetCurrentCpu
* fixups
* sceKernelGetMainSocId
Used by libSceAvPlayer to determine if console is a pro or not.
* Update process.cpp
* Set has_param_sfo to true
* Clang
* Swap write access mode for read write
Opening with access mode w will erase the opened file. We do not want this.
* Create mode
Opening with write access was previously the only way to create a file through open, so add a separate FileAccessMode that uses the write access mode to create files.
* Update file_system.cpp
Remove a hack added to posix_rename to bypass the file clearing behaviors of FileAccessMode::Write
* Check access mode in read functions
Write-only files cause the EBADF return on the various read functions. Now that we're opening files differently, properly handling this is necessary.
* Separate appends into proper modes
Fixes a potential regression from one of my prior PRs, and ensures the Write | Append flag combo also behaves properly in read-related functions.
* Move IsWriteOnly check after device/socket reads
file->f is only valid for files, so checking this before checking for sockets/devices will cause access violations.
* Fix issues
Now that Write is identical to ReadWrite, internal uses of Write need to be swapped to my new Create mode
* Fix remaining uses of FileAccessMode write to create files
Missed these before.
* Fix rebase
* Add stubbed get_authinfo (#3722)
* mostly stubbed get_authinfo
* Return value observed on console if get_authinfo was called for the current thread, esrch otherwise
---------
Co-authored-by: kalaposfos13 <153381648+kalaposfos13@users.noreply.github.com>
Co-authored-by: georgemoralis <giorgosmrls@gmail.com>
* SearchFree adjustments
* Robust address validation
I've adjusted IsValidAddress to take in a size, and check whether the whole range is contained in vma map.
If no size is provided, the function reverts to the old form of address validation instead.
* Map around gaps
As is, this should work mostly.
Only remaining issue is adding logic to pass the "mapped regions" to the guest vma map (and make such logic cross-platform).
* Initialize vma_map using gaps
This should allow memory code to catch any issues from address space gaps, and prevent non-fixed mappings from jumping to a location that isn't actually available.
* Clang
* Fix compile
* Clang
* Fix compile again
* Set system_managed_base and system_managed_size based on
Many places in our code use system_managed_base as the minimum mappable address, ensure this fact remains the same on Windows to prevent potential bugs.
* Reduce address validation in SearchFree
Allows SearchFree to function when a certain Windows GPU driver goes and reserves the whole system managed area.
Since SearchFree is only called on flexible addresses, allowing this particular case, where addresses are in bounds, but there's not enough space to map, should be safe enough.
* Bump address space size further
To handle Madden NFL 16 (and any games like it)
* More thorough logging of available memory regions
Should help with spotting weirdness.
* Formatting fixes
* Clang
* Slight reduction of user space
Still large enough to handle EA's shenanigans, but small enough that Linux doesn't break.
* Assert on VirtualQuery failure
* Extra debugging information
* Further reduce user space
This will unfix most of EA's titles, but UFC will still work.
Older windows versions support the high addresses, but trying to actually use them causes significant performance issues.
* Extra debugging info
Just in case other testers still run into issues.
* Remove debug logging
* Revert user space increases
Technically this constant is still higher than before, but weird side effects of our old logic resulted in a max address somewhere around this in main.
* address_space: Support expanded virtual memory space on macOS.
Co-Authored-By: squidbus <175574877+squidbus@users.noreply.github.com>
* Move address space constants to address_space.cpp
This ensures that all code must use the calculated address space memory values instead of the constants, since the calculated values can differ based on the platform.
This does require slight modification to thread state and gnmdriver code, since both were already using these constants directly.
* Workaround Windows 10 limitations
If a Windows 10 device is detected, use a lower value for USER_MAX to prevent system-wide hangs in VirtualAlloc2 calls.
* Fix compile for Windows-Qt
* Move tessellation_factors_ring_addr initialization to sceGnmGetTheTessellationFactorRingBufferBaseAddress
* Set image base address on Windows
This seems to work fine on Windows 11, needs testing from Windows 10 due to the previously discussed bugs.
* Set Linux executable base to 0x700000000000
This allows Linux to map the full user space without any workarounds.
Co-Authored-By: Marcin Mikołajczyk <2052578+mikusp@users.noreply.github.com>
* Basic formatting changes
* Reduce USER_MAX on Linux
Seems like finding a reliable way to move shadPS4's position in memory is difficult, for now limit the user size so we aren't trying to overwrite ourselves.
* Move memory and address_space variables.
---------
Co-authored-by: squidbus <175574877+squidbus@users.noreply.github.com>
Co-authored-by: Marcin Mikołajczyk <2052578+mikusp@users.noreply.github.com>
* Add RNG device
* rng device implementation
Tailored around libSceSsl2's usage, and based on fpPS4's implementation.
* Device file function types and log fixups
* Updated creates
Updates device file create functions to be the same as the directory file create functions.
* Fix compile
* Includes cleanup
Generally preferred to have full paths. Also removed some unused imports too.
* Fix buffer size
* Bring back cstdlib imports
Needed for Mac OS.
* Fix flag handling on Windows
Fixes a weird homebrew kalaposfos made
* Fix backing protects
Windows requires that protections on areas committed through MapViewOfFile functions are less than the original mapping.
The best way to make sure everything works is to VirtualProtect the code area with the requested protection instead of applying prot directly.
* Fix error code for sceKernelMapDirectMemory2
Real hardware returns EINVAL instead of EACCES here
* Fix prot setting in ProtectBytes
* Handle some extra protection-related edge cases.
Real hardware treats read and write as separate perms, but appends read if you call with write-only (this is visible in VirtualQuery calls)
Additionally, execute permissions are ignored when protecting dmem mappings.
* Properly handle exec permission behavior for memory pools
Calling sceKernelMemoryPoolCommit with executable permissions returns EINVAL, mprotect on pooled mappings ignores the exec protection.
* Clang
* Allow execution protection for direct memory
Further hardware tests show that the dmem area is actually executable, this permission is just hidden from the end user.
* Clang
* More descriptive assert message
* Align address and size in mmap
Like most POSIX functions, mmap aligns address down to the nearest page boundary, and aligns address up to the nearest page boundary.
Since mmap is the only memory mapping function that doesn't error early on misaligned length or size, handle the alignment in the libkernel code.
* Clang
* Fix valid flags
After changing the value, games that specify just CpuWrite would hit the error return.
* Fix prot conversion functions
The True(bool) function returns true whenever value is greater than 0. While this rarely manifested before because of our wrongly defined CpuReadWrite prot, it's now causing trouble with the corrected values.
Technically this could've also caused trouble with games mapping GpuRead permissions, but that seems to be a rare enough use case that I guess it never happened?
I've also added a warning for the case where `write & !read`, since we don't properly handle write-only permissions, and I'm not entirely sure what it would take to deal with that.
* Fix some lingering dmem issues
ReleaseDirectMemory was always unmapping with the size parameter, which could cause it to unmap too much. Since multiple mappings can reference the same dmem area, I've calculated how much of each VMA we're supposed to unmap.
Additionally, I've adjusted the logic for carving out the free dmem area to properly work if ReleaseDirectMemory is called over multiple dmem areas.
Finally, I've patched a bug with my code in UnmapMemory.
* Remove mapped dmem type
Since physical addresses can be mapped multiple times, tracking mapped pages is not necessary.
This also allows me to significantly simplify the MapMemory physical address validation logic.
* Proper implementation for sceKernelMtypeprotect
I've rewritten SetDirectMemoryType to use virtual addresses instead of physical addresses, allowing it to be used in sceKernelMtypeprotect.
To accommodate this change, I've also moved address and size alignment out of MemoryManager::Protect
* Apply memory type in sceKernelMemoryPoolCommit
* Organization
Some potentially important missing mutexes, removed some unnecessary mutexes, moved some mutexes after early error returns, and updated copyright dates
* Iterator logic cleanup
Missing end check in ClampRangeSize, and adjusted VirtualQuery and DirectMemoryQuery.
* Clang
* Adjustments
* Properly account for behavior differences in MapDirectMemory2
Undid the changes to direct memory areas, added more robust logic for changing dma types, and fixed DirectMemoryQuery to return hardware-accurate direct memory information in cases where dmas split here, but not on real hardware.
I've also changed MapMemory's is_exec flag to a validate_dmem flag, used to handle alternate behavior in MapDirectMemory2. is_exec is now determined by the use of MemoryProt::CpuExec instead.
* Clang
* Add execute permissions to physical backing
Needed for executable mappings to work properly on Windows, fixes regression in RE2 with prior commit.
* Minor variable cleanup
* Update memory.h
* Prohibit direct memory mappings with exec protections
Did a quick hardware test to confirm, only seems to be prohibited for dmem mappings though.
* Update memory.cpp
* Implement sceKernelMemoryPoolGetBlockStats
Not entirely sure on the logic behind the cached blocks work, but flushed blocks seems to just be based on committed direct memory.
* Fix comment
* Fix isDevKit
Previously, isDevKit could increase the physical memory used above the length we reserve in the backing file.
* Physical backing for flexible allocations
I took the simple approach here, creating a separate map for flexible allocations and pretty much just copying over the logic used in the direct memory map.
* Various fixups
* Fix mistake #1
* Assert + clang
* Fix 2
* Clang
* Fix CanMergeWith
Validate physical base for flexible mappings
* Clang
* Physical backing for pooled memory
* Allow VMA splitting in NameVirtualRange
This should be safe, since with the changes in this PR, the only issues that come from discrepancies between address space and vma_map are issues related to vmas being larger than address space mappings. NameVirtualRange will only ever shrink VMAs by naming part of one.
* Fix
* Fix NameVirtualRange
* Revert NameVirtualRange changes
Seems like it doesn't play nice for Windows
* Clean up isDevKit logic
We already log both isNeo and isDevKit in Emulator::Run, so the additional logging in MemoryManager::SetupMemoryRegions isn't really necessary.
I've also added a separate constant for non-pro devkit memory, as suggested.
Finally I've changed a couple constants to use the ORBIS prefix we generally follow here, instead of the SCE prefix.
* Erase flexible memory contents from physical memory on unmap
Flexible memory should not be preserved on unmap, so erase flexible contents from the physical backing when unmapping.
* Expand flexible memory map
Some games will end up fragmenting the physical backing space used for flexible memory. To reduce the frequency of this happening under normal circumstances, allocate the entirety of the remaining physical backing to the flexible memory map.
This is effectively a workaround to the problem, but at the moment I think this should suffice.
* Clang
* Fix address formatting for invalid address asserts
* Reduce event flag message to trace
This log effectively contributes nothing to debugging.
* Remove excess logging in sceKernelBatchMap
Every one of these opcodes will just log from their individual function calls anyway, and if there were issues with batch map logic, we would've known for a while now.
* Log error return during sceKernelBatchMap
May help with debugging some unstable UE games?
* Remove checks for module version_major and version_minor
Following this rule broke linking for some libraries, and introduced extra effort needed to get some homebrew running.
* Clang
* Fix rebase
* Disable libSceSsl HLE
Real hardware uses a title workaround to determine if base libSceSsl is needed. Currently, this title workaround applies to absolutely nothing.
* Implement sceKernelIsInSandbox
libSceSysmodule uses this to determine if it should use sceKernelGetFsSandboxRandomWord or just hardcode "system" for retrieving modules.
Also changes some function types to use our types.
* Update OrbisSysModule enum
Adds missing values based on library testing.
* Clang
* AF_UNIX preliminary support
* fixed windows
* added windows implemenation for socketpair
* More gotos
* added sys_socketpair for libkernel_ps2emu
---------
Co-authored-by: georgemoralis <giorgosmrls@gmail.com>
* Rename SceKernelIovec to OrbisKernelIovec
Fixes the naming scheme to match the rest of the project.
* Type fixups
Noticed this while working on the actual thing this PR is about.
* More type fixups
* Update file_system.cpp
* Abstract directory handling
* Clang
* Directory fstat
* Fix dirent loading logic
* PfsDirectory size
Seemed to be hardcoded when I ran tests, so I've hardcoded it here.
Also fixed up the reclen-aligning logic based on hardware observations.
* GetDents cleanup
Bring back the bytes < 512 error return from before, as that's still something that can be checked for out here.
I've also swapped the file type stuff to run from a switch case, so that the check for invalid file type can be used as the default case
* Support reading directories
* getdents
For PfsDirectory, getdents behaves like read does on normal directories. Since dirents are stored internally with a different struct, this means getdents has to convert the dirents before returning data.
For NormalDirectory, getdents is identical to read, so it can just call read and set the basep output.
* Directory readv
* Directory preadv
Since the file mutex is locked before these calls, messing with dirents_index like this shouldn't cause any weird side effects.
* return ORBIS_OK instead of 0
to better align with our coding standards.
* Directory lseek
* Un-modify CMakePresets.json
I keep this modified locally for Linux, but accidentally pushed it.
* Clang
* Fix mac compile
* Potential windows compile fix?
* Filename fix
* Fix normal directory d_reclen
* Comment cleanup
* Remove unnecessary dirent conversion logic
On real hardware, the records are still returned with the same reclen, despite the change in structure.
* PfsDirectory dirents_index fixes
Some weird stuff happens once you reach eof on directories.
Thankfully, PfsDirectories are rather tame in this regard.
* Change comment
* Rewrite normal directory reads
The logic for these seems to behave like a normal file, so instead of tracking a dirents_index, keep an internal buffer representing the file contents, and copy that to output buffers as needed.
* Update pfs_directory.cpp
* Clang
* Fix normal dirents
When rewriting the code, I forgot to account for the increased reclen value for the last dirent in the buffer.
* PfsDirectory::lseek fixes
Based on some additional tests, it seems like lseek unconditionally returns dirents_index, not the actual file position.
Also fixed some bugs with the logic for calculating the proper offset when games do wonky things, and fixed a potential area where games could crash.
* Downgrade stat and fstat log to debug
These functions can get pretty spammy.
* PfsDirectory: Properly track if end of file is reached
Using the metric `dirents_index < directory_content_size` fails when `directory_content_size` is larger than the actual directory size we report.
Since, from what I can tell, PfsDirectories shouldn't ever report more than 0x10000 bytes for size, this change is necessary.
* Revert "PfsDirectory: Properly track if end of file is reached"
I need to do some hardware tests to see if all this excess logic is actually necessary.
* Fix PfsDirectory directory_size
Turns out, if your game has over 1000 files in a folder, it will actually cause the size to increase.
* Update copyright date
* Move devices and directories into file_sys
I've also updated the copyright dates on all these files.
* C++ style type casts
* Remove unnecessary memset
* Use a vector for the data buffer
Simplifies logic for freeing the buffer, based on review suggestions.
* Fix potential oob array access
* Change type casts in Create function
* Clang
* std::memcpy instead of memcpy
* NormalDirectory::lseek cleanup
* Create constants for directory alignment values.
* Use const wherever possible
* Includes cleanup
* Properly align address and size in munmap
Based on observations of FreeBSD source code, fixes a Windows-related memory issue in War Thunder (CUSA00224)
* Format len and phys_addr in mmap
This should make logs slightly easier to understand, since we format these parameters in other memory calls.
* Update memory.cpp
* Various GetDents fixes
Fixed return and parameter types, and made the function return all the entries that will fit in nbytes during one call.
* Fstat dir stub changes
Changes the returned statistics to match what my PS4 tests generally show.
* Stat dir stub changes
To match my fstat changes
* select
* select for windows
* fixed windows only function
* windows error converts
* fixed up
* fixed compile
* another implementation for windows
* draft rewrite for windows
* implementation for windows
* added some debugging info
* more debugging
* extensive log
* Windows: Add device files to output fd_sets
Cyberpunk 2077 breaks without this.
Also added some formatting changes, to bring the implementation closer to our typical coding standards.
* Clang
* Formatting + cleanup
Removed some excessive logging used for debugging, and did some cleanup on the non-Windows implementation.
---------
Co-authored-by: georgemoralis <giorgosmrls@gmail.com>
Co-authored-by: Stephen Miller <millerste004@gmail.com>
* Add stubbed libkernel time functions
* Implement remaining stubbed Rtc functions
* Move Rtc from the optionally HLE loading part to the always loaded part
* Remove Rtc from the README list of LLE modules
* Mfw last second hotfix I noticed while double checking that everything's good before opening the PR
* Store platform-specific level in a separate variable
So the level logged in the getsockopt/setsockopt unreachable is actually useful in cases where the level is unknown.
* Define ORBIS_NET_IPPROTO_IPV6
Not implemented yet, but since it's known we might as well add it.
* Fix error codes
Our libSceNet code expects accurate ORBIS_NET_E* errors, while the sys_net code returns ORBIS_NET_ERROR_* errors.
* Remove duplicate getsockname implementation
* Use separate mutex for ReceivePacket calls
Calls to ReceivePacket shouldn't block other socket functions, and allowing them to block these functions frequently causes deadlocks in games that use multiple threads for socket behaviors.
That said, concurrent receives are still a potential issue, so the function should still have a mutex.
* Add missing error codes
* Clang
* Minor nit
Not sure why these were left separate from the rest of the net errnos
* Set __Error() in ConvertReturnErrorCode
Because the new error values are positive, the logic of "negative return is an error" doesn't work anymore. The easiest fix, while retaining corrected error values, is to just set __Error() in ConvertReturnErrorCode, and have that return -1 instead.
I also added some formatting fixes here too.
* Set errno on stubbed P2P socket error returns.
Otherwise the errno is just whatever was set by a previous failing function, which may cause issues in some games.
I used EAGAIN here since it appears to be valid for all three of these functions, but this can be changed if requested.
* Fix missed error returns
* Fix socket methods in file_system
Missed these
If the create flag is specified, but the file already exists, then the file should open successfully, regardless of permissions.
This fixes a crash seen in Phantasy Star Online 2 New Genesis (CUSA29813)
