Just a typical day of me pushing something a month ago, nobody testing/reviewing it, then finding out it's broken when that code inevitably makes it into production.
* Initial definitions
* internal__Fofind
* Libcinternal threads
fopen stores a valid pthread mutex in the FILE struct. Since this is exposed to the game/app, we need to handle this accurately.
* internal__Foprep (and various other functions called in it)
* Actual fopen implementation
At long last, an actual function I'm supposed to implement.
* fflush + compile fixes
* fseek implementation
Comes with functions fseek calls, aside from fflush which I pushed earlier.
* fread, _Frprep
Also changed some parameter names a tad to match how I named things in my decomp.
And fixed some bugs with how I was handling the weird offseted mode thing
* fclose, _Fofree
Not confident on this one, but we'll see I guess.
* Bug fixing
No more crashes at least, fread seems to be broken though.
* fopen bugfixes
Behavior now matches LLE, at least in how LLE font seems to use it.
* Fix _Frprep
Seems like everything works now?
* Logging
Probably going to need to swap lseek and read logs to debug/trace later but this is for debugging.
* Remove alignment check
Seems I must've misinterpreted some of what Ghidra spat out, since libSceNgs2 is calling with size 1, nmemb 4.
* Reduce fseek, fread logs to trace
* Clang
* twos
* Fixes
Still can't test properly, but this seems to hide system libs, which I'm pretty sure is the necessary difference here.
* Clang
* Extra export for sceKernelGetModuleInfo2
* Fix assert
Just a typical day of me pushing something a month ago, nobody testing/reviewing it, then finding out it's broken when that code inevitably makes it into production.
* Remove unreachable in posix_pthread_mutex_timedlock
It's apparently something that was added during pthreads rewrite, but the actual code for this function seems to be fully implemented?
Just a typical day of me pushing something a month ago, nobody testing/reviewing it, then finding out it's broken when that code inevitably makes it into production.
* added function parameters
* added logging
* more logging
* added error codes file
* sceNpWebApiCreateExtdPushEventFilter some re
* added np_web_api_internal
* more np_web_api_internal definations
* Initial types cleanup
* Basic library context handling.
Followed decomp closely, using standard library classes where possible to simplify code.
* Fix params to sceNpWebApiCreateContext
* Context logic fixes
* User contexts
* Clang
* sceNpWebApiVshInitialize
* Better initialization
* Request creation logic
* Some cleanup
* sceNpWebApiAbortRequest, sceNpWebApiDeleteRequest
* SendRequest functions
* formatting
* Update terminateContext and deleteUserContext
Addressing some unimplemented bits now that I have requests and user contexts here.
* Copyright
* sceNpWebApiCreateHandle, sceNpWebApiDeleteHandle, sceNpWebApiAbortHandle
also some bugfixing
* Extended push event filter
* abort handles in terminateContext
* Other push event filter types
* Register callbacks
* unregister callbacks
* oops
* One final update to deleteContext
* Logging changes
* Bug fixes
Fixes memory leaks, pretty sure these are the only places where that was an issue.
* sceNpWebApiCheckTimeout
* Handle and request timeouts
* Oops
* Push event filter parameters
Tested with Assassin's Creed Unity, seems to be correct.
* Service push event filter parameters
Tested again with Assassin's Creed Unity, seems to work fine.
Also fixed some code bugs I noticed, and removed an unnecessary part of my internal structs
* Stub implementation for createUserContextWithOnlineId
Might need a PSN check to be properly accurate, not sure.
* added sceNpWebApiGetHttpStatusCode
* opps
* opss 2
* sceNpWebApiReadData
* clang
* Fix context ids, user context ids, and request ids
Overlooked how these ids are actually calculated.
* Additional PSN checks
Seems creating any form of push event filter with an np service name fails when you're not connected to PSN.
Not sure of the actual cause yet, but given the error code, it's related to sceNpManagerIntGetUserList.
* compile fix
---------
Co-authored-by: Stephen Miller <millerste004@gmail.com>
Co-authored-by: Stephen Miller <56742918+StevenMiller123@users.noreply.github.com>
* improved sdl backend
* small cleanups
* misc
* adjustments and new definations
* cleanups
* more debuging
* rewrote sceAudioOut calls
* fixed a trace
* fixed audio3d port
* small debug fixes
* small additions
* using shared_ptr
* compile fixes
* make macOS happy
* using shared mutex
* implemented audio input backend
* fixed port construction based on decompile
* implemented partially sceAudioInGetSilentState to return correct code if mic device is null
* improved sdl volume handling
* dynamic volume update
* this one is for @UltraDaCat
---------
Co-authored-by: kalaposfos13 <153381648+kalaposfos13@users.noreply.github.com>
* Fix bug with "phys_addr_to_search" logic
By improperly updating the variable, some games would mark the same dmem area as mapped in different parts of the vma, and the dmem map wouldn't properly reflect the state of the vma's phys areas.
* Clang
* Oops
* Fixed sceCompanionUtilGetEvent stub
Previously we effectively stubbed with ORBIS_COMPANION_UTIL_INVALID_POINTER, which makes no sense and caused issues in games.
* Check for null issuer_id in libSceNpAuth's GetAuthorizationCode
Comes up in Mirror's Edge Catalyst, according to some debugging done by a community member.
Given the library didn't have any null checks for that value, this is probably allowed.
* PngEnc hle
* format
* formatting + fix scePngEncDelete
* fix cmake + misc improvements
i think the setjmp is right according to the libpng manual, works fine from my testing
* fixes
fix an issue with how alpha was handled, and PngEncode() now properly sets the processed_height in outputInfo.
* format
* Update pngenc.cpp
* set outputInfo->processed_height during png write
i assume some games will use this for error handling
* Avoid nullptr dereference on GetSocket
Was gonna include this in my socket PR, but that got merged before I could push this.
* Lock unmap mutex in PoolExpand and Allocate
PAYDAY 2 has a rare race condition involving dmem releases.
I'm not certain this commit will fix it, but this would cause a race condition that could cause asserts like what PAYDAY 2 can hit, so I'll just pray this does the job until I can prove it doesn't.
This issue would cause memory leaks in some EA titles, also just generally makes it harder to debug stuff when the fd table is flooded with closed sockets and epolls.
* no
no
* Adjust locking strategy
Use a separate mutex for the initial error checks + GPU unmap instead of using the reader lock. Make sure all writers lock this separate mutex, and for those that don't perform GPU unmaps, lock the writer lock immediately too.
This gets around every race condition I've envisioned so far, and hopefully does the trick?
* Clang
* Always GPU unmap
GPU unmaps have logic built-in to only run on mapped areas.
Not sure if userfaultfd would work with this, but since that's already broken anyway, I'll let reviewers decide that.
Without doing this, I'd need to do an extra pass through VMAs to find what all needs to be GPU modified before I can unmap from GPU, then perform remaining unmap work. Especially for places like MapMemory, that's a lot of code bloat.
* Fixups
* Update memory.cpp
* Rename mutex
It's really just a mutex for the sole purpose of dealing with GPU unmaps, so unmap_mutex is a bit more fitting than transition_mutex
* Optimizations
Microsoft allows you to coalesce multiple free placeholders in one VirtualFreeEx call, so we can perform the VirtualFreeEx after coalescing with neighboring regions to eliminate a VirtualFreeEx call in some situations.
* Remove unnecessary VirtualProtect call
As far as I can tell, this call wastes a bunch of time, and is completely unnecessary.
With our current codebase, simply supplying prot to MapViewOfFile3 works properly.
* Properly handle file mmaps with offsets
Pretty easy fix to perform while I'm here, so I might as well include it.
* Oops
Leftover stuff from local things + clang
* Disable tracy memory tracking
Tracy's memory tracking is built around a typical malloc/free API, so each individual alloc must correspond to a free.
Moving these to address space would fix issues on Windows, but Linux/Mac would have the same issues with our current code.
Disabling VMA merging is technically a fix, but since that's hardware-accurate behavior, I'd rather not disable it.
I'm sure there's a simple solution I'm missing, but unless other devs have a better idea of how this should be handled, the best I can do is disable it so we can keep using Tracy to trace performance.
* Update address_space.cpp
* Debug logging
Should give a decent idea of how nasty these AddressSpace calls are in games that lost perf.
* test removing thread safety
Just for testing, will revert afterwards.
* Check name before merging
Fixes a regression in Apex Legends
* Revert "test removing thread safety"
This reverts commit ab897f4b1c.
* Move mutex locks before IsValidMapping calls
These aren't thread safe, this fixes a rare race condition that I ran into with Apex Legends.
* Revert "Debug logging"
This reverts commit eb2b12a46c.
* Proper VMA splitting in ProtectBytes, SetDirectMemoryType, and NameVirtualRange
Also slight optimization by eliminating AddressSpace protect calls when requested prot matches the previous prot.
Fixes a regression in God of War: Ragnarok
* Clang
* Fixes to SetDirectMemoryType logic
Fixes some regressions in Marvel's Spider-Man that occurred with my previous commits to this PR.
* Fix Genshin Impact again
* Assert on out-of-bounds protect calls
Our page tracking code is prone to causing this.
* test mutex again
This time, remember all mutex stuff
* Revert hack
I'll work on a better way to deal with mutexes in a bit, first I'm pushing up some extra fixes
* Proper logic for checked ReleaseDirectMemory, added bounds checks
Should help some games.
* Better logging for ReleaseDirectMemory errors.
* Only perform region coalescing after all unmap operations.
A small optimization for unmapping multiple regions. Since Microsoft lets you coalesce multiple placeholders at once, we can save doing any VirtualFreeEx calls for coalescing until after we unmap everything in the requested range.
* Separate VMA creation logic into a separate method, update MapFile to use it
MapFile is technically another "emulation" of MapMemory, both should follow similar logic.
To avoid duplicating code, move shared logic to a different function that both MapMemory and MapFile can call.
This fixes memory asserts in a couple of online-only apps I have.
* Clang
* Fix TryWriteBacking
This fixes a lot of regressions that got misattributed
Co-Authored-By: TheTurtle <47210458+raphaelthegreat@users.noreply.github.com>
* Fix again
Fixes device lost crashes with some games after my last commit.
* Oops
* Mutex cleanup
Avoided changing anything in MapMemory, UnmapMemory, PoolCommit, or PoolDecommit since those all need a little extra granularity to prevent GPU deadlocking.
Everything else now uses standard library locks to make things a little simpler.
* Swap MapMemory and PoolCommit to use scoped lock
GPU maps are safe, so this is fine. Unmaps are the primary issue.
---------
Co-authored-by: TheTurtle <47210458+raphaelthegreat@users.noreply.github.com>
* Initial work
* Bug fixing
deadlocks and broken unmaps
* Fix more bugs
broken memory pools
* More bug fixing
Still plenty more to fix though
* Even more bug fixing
Finally got Final Fantasy XV back to running, haven't found anymore bugs yet.
* More bugfixing
* Update memory.cpp
* Rewrite start
* Fix for oversized unmaps
* Oops
* Update address_space.cpp
* Clang
* Mac fix?
* Track VMA physical areas based on start in VMA
Allows me to simplify some logic, and should (finally) allow merging VMAs in memory code.
* Merge VMAs, fix some bugs
Finally possible thanks to address space + phys tracking changes
* Clang
* Oops
* Oops2
* Oops3
* Bugfixing
* SDK check for coalescing
Just to rule out any issues from games that wouldn't see coalescing in the first place.
* More ReleaseDirectMemory fixes
I really suck at logic some days
* Merge physical areas within VMAs
In games that perform a lot of similar mappings, you can wind up with 1000+ phys areas in one vma.
This should reduce some of the overhead that might cause.
* Hopefully fix Mac compile
Why must their uint64_t be different?
* Mac pt.2
Oops
* Introducing key_manager for storing encryption keys . Currently only trophy key is neccesary
* keep gcc happy?
* addded logging to keymanager
* revert file
* added npbind file format and rewrote part of trp file format
* added libSceAudiodec to lle modules list
* crappy float resample , use it at your own risk
* clang
* adjustments to aac
---------
Co-authored-by: Vladislav Mikhalin <mikhalinvlad@gmail.com>
