* Swap write access mode for read write
Opening with access mode w will erase the opened file. We do not want this.
* Create mode
Opening with write access was previously the only way to create a file through open, so add a separate FileAccessMode that uses the write access mode to create files.
* Update file_system.cpp
Remove a hack added to posix_rename to bypass the file clearing behaviors of FileAccessMode::Write
* Check access mode in read functions
Write-only files cause the EBADF return on the various read functions. Now that we're opening files differently, properly handling this is necessary.
* Separate appends into proper modes
Fixes a potential regression from one of my prior PRs, and ensures the Write | Append flag combo also behaves properly in read-related functions.
* Move IsWriteOnly check after device/socket reads
file->f is only valid for files, so checking this before checking for sockets/devices will cause access violations.
* Fix issues
Now that Write is identical to ReadWrite, internal uses of Write need to be swapped to my new Create mode
* Fix remaining uses of FileAccessMode write to create files
Missed these before.
* Fix rebase
* Add stubbed get_authinfo (#3722)
* mostly stubbed get_authinfo
* Return value observed on console if get_authinfo was called for the current thread, esrch otherwise
---------
Co-authored-by: kalaposfos13 <153381648+kalaposfos13@users.noreply.github.com>
Co-authored-by: georgemoralis <giorgosmrls@gmail.com>
* Add RNG device
* rng device implementation
Tailored around libSceSsl2's usage, and based on fpPS4's implementation.
* Device file function types and log fixups
* Updated creates
Updates device file create functions to be the same as the directory file create functions.
* Fix compile
* Includes cleanup
Generally preferred to have full paths. Also removed some unused imports too.
* Fix buffer size
* Bring back cstdlib imports
Needed for Mac OS.
* Remove checks for module version_major and version_minor
Following this rule broke linking for some libraries, and introduced extra effort needed to get some homebrew running.
* Clang
* Fix rebase
* Disable libSceSsl HLE
Real hardware uses a title workaround to determine if base libSceSsl is needed. Currently, this title workaround applies to absolutely nothing.
* Rename SceKernelIovec to OrbisKernelIovec
Fixes the naming scheme to match the rest of the project.
* Type fixups
Noticed this while working on the actual thing this PR is about.
* More type fixups
* Update file_system.cpp
* Abstract directory handling
* Clang
* Directory fstat
* Fix dirent loading logic
* PfsDirectory size
Seemed to be hardcoded when I ran tests, so I've hardcoded it here.
Also fixed up the reclen-aligning logic based on hardware observations.
* GetDents cleanup
Bring back the bytes < 512 error return from before, as that's still something that can be checked for out here.
I've also swapped the file type stuff to run from a switch case, so that the check for invalid file type can be used as the default case
* Support reading directories
* getdents
For PfsDirectory, getdents behaves like read does on normal directories. Since dirents are stored internally with a different struct, this means getdents has to convert the dirents before returning data.
For NormalDirectory, getdents is identical to read, so it can just call read and set the basep output.
* Directory readv
* Directory preadv
Since the file mutex is locked before these calls, messing with dirents_index like this shouldn't cause any weird side effects.
* return ORBIS_OK instead of 0
to better align with our coding standards.
* Directory lseek
* Un-modify CMakePresets.json
I keep this modified locally for Linux, but accidentally pushed it.
* Clang
* Fix mac compile
* Potential windows compile fix?
* Filename fix
* Fix normal directory d_reclen
* Comment cleanup
* Remove unnecessary dirent conversion logic
On real hardware, the records are still returned with the same reclen, despite the change in structure.
* PfsDirectory dirents_index fixes
Some weird stuff happens once you reach eof on directories.
Thankfully, PfsDirectories are rather tame in this regard.
* Change comment
* Rewrite normal directory reads
The logic for these seems to behave like a normal file, so instead of tracking a dirents_index, keep an internal buffer representing the file contents, and copy that to output buffers as needed.
* Update pfs_directory.cpp
* Clang
* Fix normal dirents
When rewriting the code, I forgot to account for the increased reclen value for the last dirent in the buffer.
* PfsDirectory::lseek fixes
Based on some additional tests, it seems like lseek unconditionally returns dirents_index, not the actual file position.
Also fixed some bugs with the logic for calculating the proper offset when games do wonky things, and fixed a potential area where games could crash.
* Downgrade stat and fstat log to debug
These functions can get pretty spammy.
* PfsDirectory: Properly track if end of file is reached
Using the metric `dirents_index < directory_content_size` fails when `directory_content_size` is larger than the actual directory size we report.
Since, from what I can tell, PfsDirectories shouldn't ever report more than 0x10000 bytes for size, this change is necessary.
* Revert "PfsDirectory: Properly track if end of file is reached"
I need to do some hardware tests to see if all this excess logic is actually necessary.
* Fix PfsDirectory directory_size
Turns out, if your game has over 1000 files in a folder, it will actually cause the size to increase.
* Update copyright date
* Move devices and directories into file_sys
I've also updated the copyright dates on all these files.
* C++ style type casts
* Remove unnecessary memset
* Use a vector for the data buffer
Simplifies logic for freeing the buffer, based on review suggestions.
* Fix potential oob array access
* Change type casts in Create function
* Clang
* std::memcpy instead of memcpy
* NormalDirectory::lseek cleanup
* Create constants for directory alignment values.
* Use const wherever possible
* Includes cleanup
* Various GetDents fixes
Fixed return and parameter types, and made the function return all the entries that will fit in nbytes during one call.
* Fstat dir stub changes
Changes the returned statistics to match what my PS4 tests generally show.
* Stat dir stub changes
To match my fstat changes
* select
* select for windows
* fixed windows only function
* windows error converts
* fixed up
* fixed compile
* another implementation for windows
* draft rewrite for windows
* implementation for windows
* added some debugging info
* more debugging
* extensive log
* Windows: Add device files to output fd_sets
Cyberpunk 2077 breaks without this.
Also added some formatting changes, to bring the implementation closer to our typical coding standards.
* Clang
* Formatting + cleanup
Removed some excessive logging used for debugging, and did some cleanup on the non-Windows implementation.
---------
Co-authored-by: georgemoralis <giorgosmrls@gmail.com>
Co-authored-by: Stephen Miller <millerste004@gmail.com>
* Store platform-specific level in a separate variable
So the level logged in the getsockopt/setsockopt unreachable is actually useful in cases where the level is unknown.
* Define ORBIS_NET_IPPROTO_IPV6
Not implemented yet, but since it's known we might as well add it.
* Fix error codes
Our libSceNet code expects accurate ORBIS_NET_E* errors, while the sys_net code returns ORBIS_NET_ERROR_* errors.
* Remove duplicate getsockname implementation
* Use separate mutex for ReceivePacket calls
Calls to ReceivePacket shouldn't block other socket functions, and allowing them to block these functions frequently causes deadlocks in games that use multiple threads for socket behaviors.
That said, concurrent receives are still a potential issue, so the function should still have a mutex.
* Add missing error codes
* Clang
* Minor nit
Not sure why these were left separate from the rest of the net errnos
* Set __Error() in ConvertReturnErrorCode
Because the new error values are positive, the logic of "negative return is an error" doesn't work anymore. The easiest fix, while retaining corrected error values, is to just set __Error() in ConvertReturnErrorCode, and have that return -1 instead.
I also added some formatting fixes here too.
* Set errno on stubbed P2P socket error returns.
Otherwise the errno is just whatever was set by a previous failing function, which may cause issues in some games.
I used EAGAIN here since it appears to be valid for all three of these functions, but this can be changed if requested.
* Fix missed error returns
* Fix socket methods in file_system
Missed these
If the create flag is specified, but the file already exists, then the file should open successfully, regardless of permissions.
This fixes a crash seen in Phantasy Star Online 2 New Genesis (CUSA29813)
* Fix rename
We shouldn't be leaving a copy of the original filename laying around.
This fixes one of a few broken savedata checks in DRAGON BALL XENOVERSE (CUSA01341)
* sceKernelWrite hack
Seems like std::fwrite has some weird edge cases we aren't handling properly.
Until we get to the bottom of this issue, here's a hack that bypasses it.
This fixes saves in DRAGON BALL XENOVERSE (CUSA01341)
* hack fix
* Improved "hack"
* Fix rename for Windows users
Turns out, we're using copy instead of rename for a reason, and that same reason came up when adding the remove call.
Also adds a log for the sceKernelWrite issue, since that's definitely a hack that needs to be debugged.
* A real fix for the sceKernelWrite issue
Turns out, some data was just buffered.
Running Flush fixes that problem.
* Move fflush call to WriteRaw
To prevent future cases of this issue.
* Update file_system.cpp
* libSceVideodec2 struct fixes
Our code was based on an old version of the libSceVideodec2 library. Based on what I've decompiled, these structs changed somewhere around firmware 6.50, and newer versions of the library have these flexible checks to accommodate both variants of the structs.
* Static assert for AvcPictureInfo struct
All the other Videodec2 structs have static asserts, might as well use one here too.
* Initialize new values
Set proper values for frameFormat and framePitchInBytes.
`frame->linesize[0]` appears to be in bytes already, I'm not sure if that means framePitch is being set wrong though.
* Proper handling of whence 3 & 4
* Accurate directory handling in open
Directories can be opened, and can be created in open, these changes should handle that more accurately.
* Mount /app0 as read only
On real hardware, it's read only.
* Proper directory flag handling.
Even when directory is specified, it will still succeed to open non-directories.
* Check for read only directories
* Earlier ro check in posix_rmdir
Hardware tests suggest these checks are in a different order
* Clear temp folder on boot
My tests rely on this, and some games do too.
Two birds with one stone
* Clang
* Add missing DeleteHandle calls
Whoops
* Final flags adjustment in sceKernelOpen
All my current tests are now hardware accurate.
* Fix truncates
Host ftruncate consistently fails on EINVAL, I'll need to test if this issue affected Windows too.
* Windows hacks
Windows is more limiting about how folders are opened and things like that. For now, pretend these calls didn't error.
Also fixes compilation for Windows
* Final touch-ups
After expanding my test suite further, I found a couple more edge cases that needed addressing.
Bloodborne audio is still broken, I'll look into that soon.
* Remove hacky read-only behavior in posix_stat
Bloodborne apparently uses the mode parameter here when querying it's audio files, and the mode we returned led to it disabling audio entirely.
* Clang
* Cleaner code
* Combine fsync and sync flags
According to FreeBSD docs, the "sync" flag is synonymous with the fsync flag, and is only included to meet the POSIX spec.
* Log if any currently unhandled flags are encountered.
These are rare and probably not too important, but log a warning when they're seen.
* Update file_system.cpp
* Update file_system.cpp
* Clang
* Revert truncate fix
Using ftruncate works fine after moving the call to before the proper file opening code.
* Truncate before open
Open the file as read-write, then try truncating.
This fixes read | truncate flag behavior on Windows.
* Slightly adjust check for invalid flags
Any open call with invalid flags should return EINVAL, regardless of other errors parameters might cause.
* sceKernelOpen: Clean up flag handling
* sceKernelOpen: fix params
Based on decompilation, the second parameter of _open should be flags.
Additionally swaps the return and parameter types to align with our current standards.
* sceKernelOpen: Fix errors
Based on POSIX spec, if part of the path is missing, ENOENT is the correct return.
Additionally, decompilation suggests that open sets errno too.
* Fix exports
Fixes function exports to align with what they should be, based on what I've seen from decompilation and our module generator.
* Proper errno behavior on open
Since sceKernelOpen calls posix_open, errno should be set during this process.
Simplest way to handle that is to move the actual open code to posix_open and adjust error cases accordingly.
* Reorganize open calls, add error log
* Improve close
Removes the EPERM return, as it makes no sense, and swaps sceKernelClose with posix_close to properly emulate errno behavior.
* Fix log on close
* posix_open fixups
* Readd hack in posix_close
It's either this, or removing LLE DiscMap.
Or shadow implements posix sockets.
* Missing exports
Commented out while I gradually work through them all
* Remaining placeholder exports
* Swap some stuff around
I see nothing that suggests "open" only takes two parameters, so this should be completely safe.
It's also more accurate to how these are handled in libkernel, and means I won't need to reorganize anything for readv and writev.
* Update file_system.cpp
* Implement write and posix_write
* Oops
* Implement posix_readv and sceKernelReadv
Also fixes error behavior on readv, as that function shouldn't be returning any kernel error codes.
* Move sceKernelUnlink
Will deal with this one later, was just annoyed by how it's location doesn't align with the export order.
* Cleanup readv
* Implement posix_writev and sceKernelWritev
Also fixes error behavior on writev, since it shouldn't ever return kernel errors (since our device files return those)
* More cleanup on older functions
* Swap around sceKernelLseek and posix_lseek
This ensures that these have the correct error behavior, and makes their behavior align with the updated implementations for earlier functions.
* Update file_system.cpp
* Implement read
Also fixes error behavior
* Swap sceKernelMkdir and posix_mkdir
Fixes errno behavior on kernel calls, also fixed some incorrect error returns.
* Fix errno behavior on sceKernelRmdir
Also reduces function logging to bring it closer to the level of logging seen in other filesystem functions.
* Slight clean up of sceKernelStat
Fixes error behavior and changes some of the old data types.
* Refactor fstat
Fixes errno behavior, implements fstat, and shifts exports around based on file position.
Might reorganize function locations later though.
* Implement posix_ftruncate
Implements posix_ftruncate and fixes errno behavior for sceKernelFtruncate
* Add missing error conversions for more device functions
* Implement posix_rename, fix sceKernelRename errno behavior
* Add posix_preadv and posix_pread
Also fixes some incorrect error returns, fixes errno behavior, and removes an unnecessary hack.
* Fix compile
* Implement posix_getdents, getdirentries, and posix_getdirentries
Also fixes errno behavior for the kernel variants of these functions.
* Fix errno behavior of sceKernelFsync
* Implement posix_pwrite and posix_unlink
Also fixes errno behavior in related functions.
* Update file_system.cpp
* Remove SetPosixErrno
Ideally, we've handled all possible error conditions before calling these functions, so treat errors in platform-specific code as IO errors and return POSIX_EIO instead.
* Update header exports
Not sure where these get used, but might as well keep them consistent with the rest of this.
* Check if file exists before calling platform-specific code
Bloodborne checks if a file doesn't exist using open, checking if it specifically failed with error code ENOENT. To avoid working with platform-specific errnos, add a proper error return for if the file doesn't exist.
Fixes a regression in Bloodborne.
* Clang
Out of all the changes, this is apparently the only thing Clang-Format doesn't like.
I'm honestly surprised.
* Improve error checks on posix_unlink
Just because a file isn't opened doesn't mean the file doesn't exist.
Fixes the error returned if host_path.empty(), and removes the error return for when GetFile fails.
* Fix the Bloodborne fix
* Limit exports to tested functions
* More confirmed working exports
* Remaining stuff my games can test
* FS exports from firmware tests
* Bring back missing exports from main
I don't have any bootable things that call these, but since they were working well enough on main, they should be fine to readd.
* Add export for posix_pread
Spotted in Dreams a while back, might as well add it.
* Revert "Remove SetPosixErrno"
This reverts commit bdfc0c246c.
* Revert SetPosixErrno changes
shadow's using it for posix sockets, so significant modifications would introduce unnecessary merge conflicts.
* Update comment
* Add EACCES errno to SetPosixErrno
Seen in Gravity Rush.
Also reorganizes the switch case based on the posix errno value, since ordering by errno makes no sense on some OSes.
* More export fixups
Missed these during my initial pass through FS stuff because they were in kernel.cpp for some reason.
* Symbols from FS tests
Tested by messing around with firmware elfs, these atleast don't cause any crashes.
* Remove inaccurate error behavior
Seek can have offsets past the end of a file.
Also add logging for two valid whence values that are unsupported on Windows.
I'll need to verify that SEEK_HOLE and SEEK_DATA correspond to 3 and 4 respectively, I've yet to check source to verify.
* Fix error log
Oops
* Clang
Clang
* Remove close hack
Since LLE libSceDiscMap is no longer a concern, this hack shouldn't be needed.
Since sockets are still stubbed, and close can be used on sockets, I've added a warning log just in case this still occurs in some titles.
* Change SetPosixErrno unreachable to warning
I changed it to an unreachable in an earlier commit to make testing easier.
At this point, having an unreachable for this seems unnecessary, so change it to a warning instead.
* Remove Bloodborne hack
Games should be able to unlink files that aren't opened file descriptors. As far as I've tested, this doesn't break Bloodborne.
* handle getdents + fix condition + add info to description
* fix not handling dents errors
* to not overwrite it, only gather separate update entries when normal folder is done
* fix always setting entries to 0 and guest name including "UPDATE"
* reset indexes on completion
* don't use concat, fixes long standing bug
* make sce_module module loading take both paths into account
* core: Split error codes into separate files
* Reduces build times and is cleaner
* core: Bring structs and enums to codebase style
* core: More style changes
* scePthreadSetprio Changes
FindThread uses posix error codes, so the function export should apply the ORBIS wrapper to convert these. Since it uses posix codes, I've also renamed the function to align with other posix functions. Lastly, this fixes a compile warning about ret sometimes not getting initialized.
* Implement posix_munmap
Used by Hatsune Miku Project Diva X during intros. May help with stability on Linux, probably won't change anything on Windows.
* Exports
Some missing function exports I've seen in my tests.
sceKernelAvailableFlexibleMemorySize export is used in Final Fantasy XV Episode Duscae
posix_pthread_setprio and posix_pthread_getschedparam are used by Spider-Man Miles Morales
scePthreadKeyDelete is used in UE4 games.
I've also added in a typo fix related to my previous PR.
* libScePosix export for posix_pthread_attr_setguardsize
Used in Hatsune Miku Project Diva X v1.02
* libkernel: Cleanup some function places
* kernel: Refactor thread functions
* kernel: It builds
* kernel: Fix a bunch of bugs, kernel thread heap
* kernel: File cleanup pt1
* File cleanup pt2
* File cleanup pt3
* File cleanup pt4
* kernel: Add missing funcs
* kernel: Add basic exceptions for linux
* gnmdriver: Add workload functions
* kernel: Fix new pthreads code on macOS. (#1441)
* kernel: Downgrade edeadlk to log
* gnmdriver: Add sceGnmSubmitCommandBuffersForWorkload
* exception: Add context register population for macOS. (#1444)
* kernel: Pthread rewrite touchups for Windows
* kernel: Multiplatform thread implementation
* mutex: Remove spamming log
* pthread_spec: Make assert into a log
* pthread_spec: Zero initialize array
* Attempt to fix non-Windows builds
* hotfix: change incorrect NID for scePthreadAttrSetaffinity
* scePthreadAttrSetaffinity implementation
* Attempt to fix Linux
* windows: Address a bunch of address space problems
* address_space: Fix unmap of region surrounded by placeholders
* libs: Reduce logging
* pthread: Implement condvar with waitable atomics and sleepqueue
* sleepq: Separate and make faster
* time: Remove delay execution
* Causes high cpu usage in Tohou Luna Nights
* kernel: Cleanup files again
* pthread: Add missing include
* semaphore: Use binary_semaphore instead of condvar
* Seems more reliable
* libraries/sysmodule: log module on `sceSysmoduleIsLoaded`
* libraries/kernel: implement `scePthreadSetPrio`
---------
Co-authored-by: squidbus <175574877+squidbus@users.noreply.github.com>
Co-authored-by: Daniel R. <47796739+polybiusproxy@users.noreply.github.com>
* Skip destruction of adaptive mutex initializers
Based around similar behaviors implemented in the more-kernel branch. Hatsune Miku Project Diva X needs this.
* Fix posix_lseek result overflow
Seen when testing Spider-Man Miles Morales on more-kernel.
* Add posix_fsync
Used by Spider-Man Miles Morales. I've added the normal posix error handling to this function, though I'm aware sceKernelFsync doesn't return any errors currently. This is for future proofing and accuracy, as the actual libkernel does the usual error handling too.
* Properly handle VirtualQuery calls on PoolReserved memory.
* Export posix_getpagesize under libkernel
Bloons TD 5 needs this.
* Clang
* Add ElfInfo to track current game info in a singleton
* SaveData compatibility with old firmwares
* sceKernelOpen: fix for write-only mode
* imgui: add font to render non-ascii characters
* save_data: fix Backup Job including old backup in the new backup
* Save backup: fix to avoid filling the queue
Also limiting 1 backup / 10sec
* Save backup: fix search not handling empty pattern
*backup time improv
* core: Rewrite PSF parser & add encoder
add .sfo hex pattern to /scripts
* core/fs: allow to mount path as read-only
* common: Add CString wrapper to handle native null-terminated strings
* SaveData: rewrite to implement full functionality
* mock value for SYSTEM_VER
* SavaData: backup features
* SavaData: SaveDataMemory features
* imgui Ref-counted textures
- has a background thread to decode textures
* imgui: rework gamepad navigation
* PSF: fixed psf not using enum class for PSFEntryFmt (was a standard old ugly enum)
- Add null check to CString when itself is used in a nullable field
* SaveDataDialog implementation
- Fix Mounting/Unmounting check of SaveInstance
* libSceRazorCpu needed for gravity rush
* libSceCesCs needed for blue reflection
* clang format fix
* set scePadSetVibration to log debug
* initial sceNetCtl module implementation
* improved error codes in file system
* some intial work on netctl callbacks (helps a bit CUSA10135)
* misc
* improved callbacks handling in sceNetCtl
* small fixes
* added libSceRudp.sprx to lle modules
* draft work for npcallbacks
fix: file name typo constant_propogation_pass.cpp
fix typo from 'symbol_vitrual_addr' variable
fix typo in emit_spirv_context_get_set.cpp
fix typo from constant_propagation_pass.cpp in CMakeLists
fix typo in these some config.cpp functions
- setSliderPosition
- setSliderPositionGrid
- getSliderPosition
- getSliderPositionGrid
fix typo inside src\core\aerolib\stubs.cpp
fix typo in a comment from src\core\file_format\pkg.cpp
fix typo inside src\core\file_sys\fs.cpp + fs.h
- NeedsCaseInsensiveSearch -> NeedsCaseInsensitiveSearch
fix 2 function typos: sceAppContentAddcontEnqueueDownloadByEntitlemetId and sceAppContentAddcontMountByEntitlemetId
fix typo on comment inside src\core\libraries\kernel\file_system.cpp
fix typo on src\core\libraries\videoout\driver.cpp
fix typo in src\core\memory.cpp
fix typo from comment in src\qt_gui\game_list_utils.h
fix typo in src\video_core\amdgpu\liverpool.h
- window_offset_disble to window_offset_disable
fix typo from comments in src\video_core\host_shaders\detile_m32x1.comp + detile_m32x2.comp
- subotimal -> suboptimal
fix typo from comment in src\video_core\renderer_vulkan\renderer_vulkan.cpp
- dimentions -> dimensions
fix typo from enum in src\common\debug.h and other files
- MarkersPallete -> MarkersPalette
fix last typo in src\video_core\amdgpu\pm4_opcodes.h
- PremableCntl -> PreambleCntl
* gui: add option to boot a game by choosing elf file
* core: some small implementations
* fs: implement open func
* add some validations
* spirv: add image format
* video_core: add eR16Uint to formats
