Proposals / #951

Mac D2COMP_ValidateCOFLayerData: __fastcc void * (byte * pCofData, int32_t nLayerIndex, int32_t nFrameIndex)

applied · by KierTaylor · submitted 2026-11-02, applied 2026-11-02

Ghidra vmac 111 · 3c6740e

Mac body: ECX (ESI) is null-checked (line 0x400) and is the COF blob base; EDX (EDI) is multiplied by header byte [1] (layer stride) and [EBP+8] is added with +9, the result scaled by header byte [0], plus [1] + base + 0x1c, then non-null-asserted (line 0x408) and returned. That is Windows D2COMP_ValidateCOFLayerData 0x005032b0's formula base + ((hdr[1]*nLayerIndex + 9 + nFrameIndex) * hdr[0]) + hdr[1] + 0x1c, so EDX = nLayerIndex and the stack slot = nFrameIndex; the blob comes first on Mac. (pb prefix refused, so pCofData.)

Changes 1, applied together

1. Set prototype mac 0x0003f458

   signatureundefined D2COMP_ValidateCOFLayerData(void)→void * D2COMP_ValidateCOFLayerData(byte * pCofData, int32_t nLayerIndex, int32_t nFrameIndex)
   conventionunknown→__fastcc