Skip to content

feat(libc): implement collate<wchar_t>::_Getcat - #1982

Closed
overlordxrz-source wants to merge 4 commits into
boykopovar:mainfrom
overlordxrz-source:feat/libc-wide-collate-atomic-assert
Closed

overlordxrz-source wants to merge 4 commits into
boykopovar:mainfrom
overlordxrz-source:feat/libc-wide-collate-atomic-assert

Conversation

@overlordxrz-source

@overlordxrz-source overlordxrz-source commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

What

_ZNSt7collateIwE7_GetcatEPPKNSt6locale5facetEPKS1_ (std::collate<wchar_t>::_Getcat) still throws on main. Dreaming Sarah (PPSA02929, import list in #265) imports it.

std::_Atomic_assert has been taken out of this PR. Cyberpunk 2077 imports it but never calls it, so its stub stays as it is on main.

The wide facet mirrors the existing collate<char>::_Getcat from #885. That PR read the narrow facet's layout from the console's libc.prx (the copy in Astro's Playroom). Nobody has disassembled the wide one, so its layout here comes from the fact that both facets are instances of the same Dinkumware templates. #885 found that the narrow facet behaves the way MS STL's sources of the same line do: FNV-1a in do_hash, a 16-byte inline string buffer, and the p[-1] large-allocation header at 0x1000 bytes. Each point below is the _Elem = wchar_t form of that template, plus a source for it:

  • Facet and vtable. collate<_Elem> is one class template: MS STL <locale>. Its virtuals are the destructor, then do_compare, do_transform and do_hash, in the same order for every _Elem. So the wide facet has the narrow facet's 0x20-byte object and its vtable slot offsets (0x20/0x28/0x30). Only the element type of the arguments changes.
  • Element type. wchar_t is 16-bit unsigned on PS4 and PS5: clang's PSOSTargetInfo sets WCharType = UnsignedShort (OSTargets.h).
  • do_compare. In the C locale, Dinkumware's wcscoll has no collation table (_WCostate is {{0}}, "1-to-1 inline": xwcostate.c). So _CWcsxfrm copies the characters unchanged (xwcsxfrm.c), and _Wcscollx compares them with wmemcmp (wcscoll.c). wmemcmp returns -1/+1 on the first differing wchar_t (wmemcmp.c). The result is the order of the unsigned 16-bit values, with the shorter range first on a tie. This is the wide form of the narrow strcoll/strcmp result in feat(libc): implement collate<char>::_Getcat #885.
  • do_transform. The C-locale transform is the same one-to-one copy. The result is a guest std::wstring. basic_string keeps the narrow layout (proxy word at 0x0, buffer/pointer union at 0x8, size at 0x18, capacity at 0x20), and its inline buffer is _BUF_SIZE = 16 / sizeof(value_type) = 8 elements, so 7 characters plus the terminator (xstring). The over-aligned large-allocation path starts at _Big_allocation_threshold = 4096 bytes (xmemory). That is 2047 wide characters plus the terminator, so those results throw, like the 4095-character limit on the narrow side.
  • do_hash. MS STL hashes count * sizeof(_Elem) bytes with FNV-1a (_Hash_array_representation, called from collate::do_hash). For wchar_t on x86-64 those are the little-endian bytes of each character. For char this gives the FNV-1a that feat(libc): implement collate<char>::_Getcat #885 saw on the console.

What this does not establish: the wide instance has not been disassembled or traced on a console, so these points are inferred from the template and not measured. TechnicalDebt says so, along with the C-locale-only and null-table limits it shares with the narrow facet. The static_asserts in GuestLocale.hpp only keep the struct in line with the offsets above. They are not evidence.

#1498 also changes this stub as part of a larger change, and the two conflict in LocaleSupport.cpp.

Log

I don't own Dreaming Sarah, so I can't paste a stop line from main. The only evidence that a title reaches this export is the import list in #265.

Tested

aarch64 Debian 13 host. The full project can't be built here because it needs x86-64.

  • Extended core/libs/tests/GuestCollate.cpp covers the wide facet: _Getcat creation and reuse, compare ordering and prefix cases, transform inline and heap results, and hash equality. It passes in a native build of GuestCollate.cpp, LocaleSupport.cpp and ApplicationHeap.cpp under UBSan.
  • LocaleSupport.cpp, Export.cpp and the test compile for x86_64-linux-gnu and x86_64-windows-gnu with -Wall -Wextra (Zig 0.17 clang). The only warning is the existing -Wreturn-type-c-linkage from General.hpp.
  • python3 tools/check_conventions.py --base origin/main passes.
  • Not run with a title.

Checklist

  • Based on current main. The one other open PR that changes this function is feat(libs): implement font, libc, libkernel and misc prx stubs #1498, which also changes this stub as part of a larger change; see What
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Progress report (f145139 → d5d31d5)

📈 System libraries: 84.81% (+0.05%, +4 implemented functions)

✅ 4 implemented
Library Name
libc WideCollateCompare
libc WideCollateHash
libc WideCollateTransform
libc _ZNSt7collateIwE7_GetcatEPPKNSt6locale5facetEPKS1__nid_postfix

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The local wide-collate tests are detailed and CI now passes, but the PR only has import-list evidence and explicitly says it has not been run with either title. CONTRIBUTING.md says import lists alone do not establish that a title reaches these exports and asks for a reproduced failure or another concrete demonstrated need. The central wide-string layout/vtable and hash assumptions also need a source or runtime trace; static_asserts only check the proposed layout against itself. Finally, TechnicalDebt currently says Cyberpunk imports _Atomic_assert without calling it. Please add evidence for a reached export and ground the wide ABI/behavior before review.

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
#1832 docs/dev/TechnicalDebt.md L347-348

Cyberpunk 2077 imports std::_Atomic_assert without calling it, so the export keeps throwing until a title is shown to reach it.
@overlordxrz-source overlordxrz-source changed the title feat(libc): implement collate<wchar_t>::_Getcat and std::_Atomic_assert feat(libc): implement collate<wchar_t>::_Getcat Oct 9, 2026
@overlordxrz-source

Copy link
Copy Markdown
Contributor Author

Thanks. I made three changes:

  • _Atomic_assert: removed from this PR. Cyberpunk imports it without calling it, so the stub stays as it is on main, along with its comment. The TechnicalDebt line is gone (febc858c). The title is now feat(libc): implement collate<wchar_t>::_Getcat.
  • Wide layout, vtable and hash: you're right that the static_asserts only check the struct against itself. Nobody has disassembled or traced the wide instance. The description now cites the sources it follows. feat(libc): implement collate<char>::_Getcat #885 read the narrow facet from the console's libc.prx and found it matches MS STL's Dinkumware line, and the wide facet is the _Elem = wchar_t instance of the same templates. That gives the vtable order from collate<_Elem>, the 8-element inline buffer from _BUF_SIZE = 16 / sizeof(value_type), and the 4096-byte large-allocation threshold. The hash is FNV-1a over count * sizeof(_Elem) bytes. In the C locale, Dinkumware's wcscoll has no table and does a one-to-one copy plus wmemcmp. PS4 and PS5 wchar_t is unsigned 16-bit per clang's PSOSTargetInfo. TechnicalDebt now says plainly that the wide facet is inferred from these sources and not checked against the console (02c2bd0d).
  • Evidence of a reached export: I still only have the import list in docs(nids): announce Dreaming Sarah PPSA02929 missing imports #265, with no main log. If that doesn't meet CONTRIBUTING.md, please convert this to a draft until someone with Dreaming Sarah can post the stop line. I'd rather that than argue the rule.

@oneandonlydean

Copy link
Copy Markdown
Collaborator

Closing: there's no stop line from main for collate<wchar_t>::_Getcat, only the import list in #265, and the wide layout hasn't been checked against the console. Reopen when it's a title's next stop on main, with the log line.

ycbuzi pushed a commit to ycbuzi/AnyPS5 that referenced this pull request Oct 10, 2026
…ayout is taken from

Harvested from upstream (closed unmerged); library completion 2831/3229 -> 2835/3232. (changelog conflict: kept both)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011krDVCnnMQPaEFEqCDnKmA
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants