Just a typical day of me pushing something a month ago, nobody testing/reviewing it, then finding out it's broken when that code inevitably makes it into production.
* added function parameters
* added logging
* more logging
* added error codes file
* sceNpWebApiCreateExtdPushEventFilter some re
* added np_web_api_internal
* more np_web_api_internal definations
* Initial types cleanup
* Basic library context handling.
Followed decomp closely, using standard library classes where possible to simplify code.
* Fix params to sceNpWebApiCreateContext
* Context logic fixes
* User contexts
* Clang
* sceNpWebApiVshInitialize
* Better initialization
* Request creation logic
* Some cleanup
* sceNpWebApiAbortRequest, sceNpWebApiDeleteRequest
* SendRequest functions
* formatting
* Update terminateContext and deleteUserContext
Addressing some unimplemented bits now that I have requests and user contexts here.
* Copyright
* sceNpWebApiCreateHandle, sceNpWebApiDeleteHandle, sceNpWebApiAbortHandle
also some bugfixing
* Extended push event filter
* abort handles in terminateContext
* Other push event filter types
* Register callbacks
* unregister callbacks
* oops
* One final update to deleteContext
* Logging changes
* Bug fixes
Fixes memory leaks, pretty sure these are the only places where that was an issue.
* sceNpWebApiCheckTimeout
* Handle and request timeouts
* Oops
* Push event filter parameters
Tested with Assassin's Creed Unity, seems to be correct.
* Service push event filter parameters
Tested again with Assassin's Creed Unity, seems to work fine.
Also fixed some code bugs I noticed, and removed an unnecessary part of my internal structs
* Stub implementation for createUserContextWithOnlineId
Might need a PSN check to be properly accurate, not sure.
* added sceNpWebApiGetHttpStatusCode
* opps
* opss 2
* sceNpWebApiReadData
* clang
* Fix context ids, user context ids, and request ids
Overlooked how these ids are actually calculated.
* Additional PSN checks
Seems creating any form of push event filter with an np service name fails when you're not connected to PSN.
Not sure of the actual cause yet, but given the error code, it's related to sceNpManagerIntGetUserList.
* compile fix
---------
Co-authored-by: Stephen Miller <millerste004@gmail.com>
Co-authored-by: Stephen Miller <56742918+StevenMiller123@users.noreply.github.com>
* improved sdl backend
* small cleanups
* misc
* adjustments and new definations
* cleanups
* more debuging
* rewrote sceAudioOut calls
* fixed a trace
* fixed audio3d port
* small debug fixes
* small additions
* using shared_ptr
* compile fixes
* make macOS happy
* using shared mutex
* implemented audio input backend
* fixed port construction based on decompile
* implemented partially sceAudioInGetSilentState to return correct code if mic device is null
* improved sdl volume handling
* dynamic volume update
* this one is for @UltraDaCat
---------
Co-authored-by: kalaposfos13 <153381648+kalaposfos13@users.noreply.github.com>
* Fix bug with "phys_addr_to_search" logic
By improperly updating the variable, some games would mark the same dmem area as mapped in different parts of the vma, and the dmem map wouldn't properly reflect the state of the vma's phys areas.
* Clang
* Oops
* Fixed sceCompanionUtilGetEvent stub
Previously we effectively stubbed with ORBIS_COMPANION_UTIL_INVALID_POINTER, which makes no sense and caused issues in games.
* Check for null issuer_id in libSceNpAuth's GetAuthorizationCode
Comes up in Mirror's Edge Catalyst, according to some debugging done by a community member.
Given the library didn't have any null checks for that value, this is probably allowed.
SetThreadName gets passed an std::string's c_str whose pointer gets invalidated by the assignment of g_curthread->name, resulting in broken thread names further down the line
* PngEnc hle
* format
* formatting + fix scePngEncDelete
* fix cmake + misc improvements
i think the setjmp is right according to the libpng manual, works fine from my testing
* fixes
fix an issue with how alpha was handled, and PngEncode() now properly sets the processed_height in outputInfo.
* format
* Update pngenc.cpp
* set outputInfo->processed_height during png write
i assume some games will use this for error handling
* Avoid nullptr dereference on GetSocket
Was gonna include this in my socket PR, but that got merged before I could push this.
* Lock unmap mutex in PoolExpand and Allocate
PAYDAY 2 has a rare race condition involving dmem releases.
I'm not certain this commit will fix it, but this would cause a race condition that could cause asserts like what PAYDAY 2 can hit, so I'll just pray this does the job until I can prove it doesn't.
This issue would cause memory leaks in some EA titles, also just generally makes it harder to debug stuff when the fd table is flooded with closed sockets and epolls.
* no
no
* Adjust locking strategy
Use a separate mutex for the initial error checks + GPU unmap instead of using the reader lock. Make sure all writers lock this separate mutex, and for those that don't perform GPU unmaps, lock the writer lock immediately too.
This gets around every race condition I've envisioned so far, and hopefully does the trick?
* Clang
* Always GPU unmap
GPU unmaps have logic built-in to only run on mapped areas.
Not sure if userfaultfd would work with this, but since that's already broken anyway, I'll let reviewers decide that.
Without doing this, I'd need to do an extra pass through VMAs to find what all needs to be GPU modified before I can unmap from GPU, then perform remaining unmap work. Especially for places like MapMemory, that's a lot of code bloat.
* Fixups
* Update memory.cpp
* Rename mutex
It's really just a mutex for the sole purpose of dealing with GPU unmaps, so unmap_mutex is a bit more fitting than transition_mutex
* Optimizations
Microsoft allows you to coalesce multiple free placeholders in one VirtualFreeEx call, so we can perform the VirtualFreeEx after coalescing with neighboring regions to eliminate a VirtualFreeEx call in some situations.
* Remove unnecessary VirtualProtect call
As far as I can tell, this call wastes a bunch of time, and is completely unnecessary.
With our current codebase, simply supplying prot to MapViewOfFile3 works properly.
* Properly handle file mmaps with offsets
Pretty easy fix to perform while I'm here, so I might as well include it.
* Oops
Leftover stuff from local things + clang
* Disable tracy memory tracking
Tracy's memory tracking is built around a typical malloc/free API, so each individual alloc must correspond to a free.
Moving these to address space would fix issues on Windows, but Linux/Mac would have the same issues with our current code.
Disabling VMA merging is technically a fix, but since that's hardware-accurate behavior, I'd rather not disable it.
I'm sure there's a simple solution I'm missing, but unless other devs have a better idea of how this should be handled, the best I can do is disable it so we can keep using Tracy to trace performance.
* Update address_space.cpp
* Debug logging
Should give a decent idea of how nasty these AddressSpace calls are in games that lost perf.
* test removing thread safety
Just for testing, will revert afterwards.
* Check name before merging
Fixes a regression in Apex Legends
* Revert "test removing thread safety"
This reverts commit ab897f4b1c.
* Move mutex locks before IsValidMapping calls
These aren't thread safe, this fixes a rare race condition that I ran into with Apex Legends.
* Revert "Debug logging"
This reverts commit eb2b12a46c.
* Proper VMA splitting in ProtectBytes, SetDirectMemoryType, and NameVirtualRange
Also slight optimization by eliminating AddressSpace protect calls when requested prot matches the previous prot.
Fixes a regression in God of War: Ragnarok
* Clang
* Fixes to SetDirectMemoryType logic
Fixes some regressions in Marvel's Spider-Man that occurred with my previous commits to this PR.
* Fix Genshin Impact again
* Assert on out-of-bounds protect calls
Our page tracking code is prone to causing this.
* test mutex again
This time, remember all mutex stuff
* Revert hack
I'll work on a better way to deal with mutexes in a bit, first I'm pushing up some extra fixes
* Proper logic for checked ReleaseDirectMemory, added bounds checks
Should help some games.
* Better logging for ReleaseDirectMemory errors.
* Only perform region coalescing after all unmap operations.
A small optimization for unmapping multiple regions. Since Microsoft lets you coalesce multiple placeholders at once, we can save doing any VirtualFreeEx calls for coalescing until after we unmap everything in the requested range.
* Separate VMA creation logic into a separate method, update MapFile to use it
MapFile is technically another "emulation" of MapMemory, both should follow similar logic.
To avoid duplicating code, move shared logic to a different function that both MapMemory and MapFile can call.
This fixes memory asserts in a couple of online-only apps I have.
* Clang
* Fix TryWriteBacking
This fixes a lot of regressions that got misattributed
Co-Authored-By: TheTurtle <47210458+raphaelthegreat@users.noreply.github.com>
* Fix again
Fixes device lost crashes with some games after my last commit.
* Oops
* Mutex cleanup
Avoided changing anything in MapMemory, UnmapMemory, PoolCommit, or PoolDecommit since those all need a little extra granularity to prevent GPU deadlocking.
Everything else now uses standard library locks to make things a little simpler.
* Swap MapMemory and PoolCommit to use scoped lock
GPU maps are safe, so this is fine. Unmaps are the primary issue.
---------
Co-authored-by: TheTurtle <47210458+raphaelthegreat@users.noreply.github.com>
* Initial work
* Bug fixing
deadlocks and broken unmaps
* Fix more bugs
broken memory pools
* More bug fixing
Still plenty more to fix though
* Even more bug fixing
Finally got Final Fantasy XV back to running, haven't found anymore bugs yet.
* More bugfixing
* Update memory.cpp
* Rewrite start
* Fix for oversized unmaps
* Oops
* Update address_space.cpp
* Clang
* Mac fix?
* Track VMA physical areas based on start in VMA
Allows me to simplify some logic, and should (finally) allow merging VMAs in memory code.
* Merge VMAs, fix some bugs
Finally possible thanks to address space + phys tracking changes
* Clang
* Oops
* Oops2
* Oops3
* Bugfixing
* SDK check for coalescing
Just to rule out any issues from games that wouldn't see coalescing in the first place.
* More ReleaseDirectMemory fixes
I really suck at logic some days
* Merge physical areas within VMAs
In games that perform a lot of similar mappings, you can wind up with 1000+ phys areas in one vma.
This should reduce some of the overhead that might cause.
* Hopefully fix Mac compile
Why must their uint64_t be different?
* Mac pt.2
Oops
* Introducing key_manager for storing encryption keys . Currently only trophy key is neccesary
* keep gcc happy?
* addded logging to keymanager
* revert file
* added npbind file format and rewrote part of trp file format
When walking the users of special constants which form LDS
addresses:
-ignore when a user contributes to the wrong operand of an LDS inst, for
example the data operand of WriteShared* instead of the address operand.
This can mistakenly happen due to phi nodes.
-don't use flags to stash temp info about phis, since flags may already
be in use. Use a separate map.
* added libSceAudiodec to lle modules list
* crappy float resample , use it at your own risk
* clang
* adjustments to aac
---------
Co-authored-by: Vladislav Mikhalin <mikhalinvlad@gmail.com>
* OrbisAudio3dOpenParameters struct fix
Not sure why we have the extra filler, but decomp suggests it shouldn't exist.
This fixes stack_chk_fail issues in audio3d using titles.
* Bring back filler, only copy 0x20 bytes.
The library accepts variations on struct size, with the maximum size being the 0x28 size our current struct has.
This fixes the issue without potentially breaking the struct.
* Fix memcpy
Prevent OOB read
* Run libSceRtc LLE
The more we've used our HLE, the more issues we've had with it.
While we debug these bugs, re-enabling LLE will address any regressions the swap caused.
* libSceJpegDec LLE
Needed for Trackmania until we implement HLE for this library
* libScePngEnc LLE
Needed for Minecraft until we implement HLE for this library
* Update documentation appropriately
* libSceJpegEnc LLE
By @georgemoralis's request
