1
0
Fork 0
mirror of https://github.com/zeldaret/oot.git synced 2025-08-08 07:20:16 +00:00
oot/tools/assets/descriptor/spec.md
Dragorn421 1e556e3a3d
New assets system (#2481)
* wip: New assets system tm

Builds gc-eu-mq-dbg OK from clean after
1) make setup
2) python3 -m tools.assets.extract -j
3) replace 0x80A8E610 with sShadowTex in extracted/gc-eu-mq-dbg/assets/overlays/ovl_En_Jsjutan/sShadowMaterialDL.inc.c
4) make various symbols in extracted data like sTex static

* use variables from config.yml for gMtxClear and sShadowTex addresses

* Write source with static for overlays using `HACK_IS_STATIC_ON` hack

* gc-eu-mq-dbg OK from clean with `make setup && make`

* implement more skeleton-related types, cleanups, fixups

* fix extracted data to no longer produce compilation warnings

* implement more of RoomShapeImage types

* yeet XmlPath from ExternalFile usage

* Implement PlayerAnimationDataResource (link_animetion data)

* fix csdis CS_TIME extra arg

* dmadata file names no longer hardcoded for gc-eu-mq-dbg

* ntsc-1.0 OK

* xml fixes

* slightly improve standard output

* rm extract_assets.py

* generate and use Limb enums (TODO: check Skin skels and implement for Curve skels)

* handle dependencies between xmls

* introduce RawPointers xml attribute to ignore specific pointers and keep them raw

* add tools/extract_assets.sh

* fixups

* only extract if xmls changed or if -f (force) is used

* fixups, gc-eu OK

* all versions OK

* check attributes of xml resources elements

* Implement legacy skelanime resources

* fix ASSET_FILES_BIN_EXTRACTED/COMMITTED: look for .u8.bin specifically instead of just .bin

* implement JFIFResource

* fix png/jpg wildcards: look specifically for .u64.png .u32.png .u64.jpg

* Makefile: Add rules to build .png, .bin and .jpg in assets/ too

* start writing actual docs

* extract sTransCircleDL and sTransWipeDL

* misc cleanup/fixes, pygfxd 1.0.3

* refactor CDataExt.set_write callback args to use a dataclass

* Move {} to in-source

* misc

* more progress on spec

* fix missing braces in n64dd_error_textures.c

* finish xml spec doc

* assets xmls fixes

* some cleanup, use `gNameTex_WIDTH/HEIGHT` macros in dlists

* handle hackmode_syotes_room, fix compile

* C build_from_png

* rm tools/assets/bin2c

* rm ZAPD

* format

* remove rule to generate dmadata_table.py

* CC0 license (and some import cleanup)

* dont try to build zapd (rmd)

* simplify palettes with single user (ci images with a non-shared palette)

* add docs on how images are handled

* bss

* allow -j N

* fix n64texconv python bindings memory management

* move -j at the end of calling extraction script

* with -j, update last_extracts.json as each job completes rather than only if all complete

* make interrupting less jank by making child processes ignore sigint

* use enum names in `SCENE_CMD_SKYBOX_SETTINGS`

* `multiprocessing.get_context("fork")`

* import rich, except ImportError s

* fix optional rich usage

* .bss

* .bss

* .bss

* assets extraction: -j -> -j$(N_THREADS)

* .bss

* change LIMB_NONE/MAX defaults to be FILE_OFFSET instead of SKELNAME

* 0XHEX -> 0xHEX

* fix bss

* Proper includes for assets

mostly proper, some includes like dlists resources always causing a sys_matrix.h include (when not every dlist references gIdentityMtx) could be done better

* rm z64.h

* rm z64.h take two

* bss

* Make .u64 suffix for pngs optional

* fixup: rm .u64 suffix from n64dd image paths

* Remove elemtype suffixes from .bin and .jpg files

* Update images.md

* some build_from_png cleanup, more error handling, comments

* Handle skybox textures

Introduce "sub-format" suffix for pngs, with sub-formats split_lo and split_hi being used for skybox textures

* fixup for older python

* improve collision output some

* fully use SURFACETYPE[01] macros in writing extracted surface types

* use WATERBOX_PROPERTIES in extracted waterboxes

* some SceneCommandsResource cleanup

* format EnvLightSettingsList output
2025-05-18 01:29:09 +02:00

6.8 KiB

This document describes the expected structure of xml files describing assets.

Top elements

Root

<Root>
    ...
</Root>

This is the root element in the file, containing exclusively <File> and <ExternalFile> elements as direct children.

File

<File Name="baserom_file" Segment="11">
    ...
</File>

A <File> contains resources elements as children.

  • Required attributes: Name
  • Optional attributes: Segment

Name is the name of the baserom file from which the data is to be extracted.

Segment (decimal) is the segment number for the file.

ExternalFile

<ExternalFile OutPath="assets/name"/>

Declare the <File>s in the xml may reference symbols from an external file.

The external file is located by matching its name against the list of assets in the version's config.yml.

For example, baseroms/gc-eu/config.yml contains

assets:
- name: objects/gameplay_keep
  xml_path: assets/xml/objects/gameplay_keep_pal.xml

then <ExternalFile OutPath="assets/objects/gameplay_keep/"/> refers to that gameplay_keep entry, which uses the gameplay_keep_pal.xml xml file when extracting assets for version gc-eu.

Resource elements

Resource elements describe resources. Resources are pieces of data corresponding to a symbol each.

Two attributes are required on all resource elements: Name and Offset.

  • Name is the name of the symbol associated to the resource.
  • Offset is the location in bytes from the start of the file data.

Blob

<Blob Name="gNameBlob" Size="0x421" Offset="0x1230">

Unstructured binary data.

  • Required attributes: Size

Size is the size of the binary blob in bytes.

DList

<DList Name="gNameDL" Offset="0x1230" Ucode="f3dex2" RawPointers="0x08000000,0x09000000"/>

A display list.

  • Optional attributes: Ucode, RawPointers

Ucode (defaults to f3dex2) picks the graphics microcode for which to disassemble the dlist. It may be f3dex or f3dex2.

RawPointers (defaults to an empty value) is a comma-separated list of values the display list uses as raw pointers ("hex" instead of a symbol). The purpose of this attribute is to silence extraction warnings.

Mtx

<Mtx Name="gNameMtx" Offset="0x1230"/>

A fixed-point matrix.

Texture

<Texture Name="gNameTex" Format="rgba16" Width="16" Height="16" Offset="0x1230"/>
<Texture Name="gNameTex" Format="ci8" Width="16" Height="16" Offset="0x1230" TlutOffset="0x2340"/>
<Texture Name="gNameTex" Format="ci8" Width="16" Height="16" Offset="0x1230" ExternalTlut="baserom_file" ExternalTlutOffset="0x2340"/>

A texture, an image in one of the native N64 formats.

  • Required attributes for all formats: Format, Width, Height
  • Required attributes for CI formats (ci4, ci8): TlutOffset, or ExternalTlut and ExternalTlutOffset

Format is the format of the texture, one of rgba32, rgba16, i4, i8, ia4, ia8, ia16, ci4 or ci8.

Width and Height specify the dimensions of the texture.

For CI formats, the TLUT (Texture Look Up Table, or palette) must be specified with either TlutOffset if the TLUT is in the same file as the texture, or both of ExternalTlut and ExternalTlutOffset if the TLUT is in a different file. ExternalTlut is the name of the baserom file where the TLUT is. In both cases, the TLUT must also be declared as a resource.

Array

<Array Name="gNameVtx" Count="42" Offset="0x1230">
    <Vtx/>
</Array>
<Array Name="gNameVec3sArray" Count="42" Offset="0x1230">
    <Vector Type="s16" Dimensions="3"/>
</Array>
<Array Name="gNameS16Array" Count="42" Offset="0x1230">
    <Scalar Type="s16"/>
</Array>

An array of vertices, vectors or scalars. The child element determines the array's element type.

  • Required attributes: Count

Count is the length of the array.

The child element may be one of <Vtx/> (for Vtx[]), <Vector Type="s16" Dimensions="3"/> (for Vec3s[]) or <Scalar Type="s16"> (for s16[]).

Scene

<Scene Name="name" Offset="0x1230"/>

Scene commands.

Room

<Room Name="name" Offset="0x1230"/>

Room commands.

Collision

<Collision Name="gNameCol" Offset="0x1230"/>

Collision header.

Cutscene

<Cutscene Name="gNameCs" Offset="0x1230"/>

Cutscene script.

Path

<Path Name="gNamePathList" Offset="0x1230" NumPaths="12"/>

Path list.

  • Required attributes: NumPaths

NumPaths is the length of the path list.

Skeleton

<Skeleton Name="gNameSkel" Type="Normal" LimbType="Standard" LimbNone="NAME_LIMB_NONE" LimbMax="NAME_LIMB_MAX" EnumName="NameLimb" Offset="0x1230"/>

Skeleton header.

  • Required attributes: Type, LimbType
  • Optional attributes: LimbNone, LimbMax, EnumName

Type is the type of the skeleton, one of Normal, Flex or Curve.

LimbType is the type of limb used in the skeleton, one of Standard, LOD, Skin, Curve or Legacy.

Not all skeleton types are compatible with all limb types:

LimbType Compatible skeleton Type
Standard Normal, Flex
LOD Normal, Flex
Skin Normal
Curve Curve
Legacy none

LimbNone, LimbMax, EnumName can be set to override the corresponding names in the generated limb enum:

typedef enum NameLimb {
    NAME_LIMB_NONE,
    ...
    NAME_LIMB_MAX
} NameLimb;

LimbTable

<LimbTable Name="gNameLimbs" LimbType="Standard" Count="12" Offset="0x1230"/>

Limb table.

  • Required attributes: LimbType, Count

LimbType is one of Standard, LOD, Skin, Curve or Legacy.

Count is the amount of limbs.

Limb

<Limb Name="gNameLimb" LimbType="Standard" EnumName="NAME_LIMB_NAME" Offset="0x1230"/>

Limb of a skeleton.

  • Required attributes: LimbType
  • Optional attributes: EnumName

LimbType is one of Standard, LOD, Skin, Curve or Legacy.

EnumName can be set to override the limb name in the generated limb enum.

Animation

<Animation Name="gNameAnim" Offset="0x1230"/>

Animation header.

CurveAnimation

<CurveAnimation Name="gNameAnim" SkelOffset="0x120" Offset="0x1230"/>

Curve animation header.

  • Required attributes: SkelOffset

SkelOffset is the offset of the skeleton which uses this animation. The skeleton must also be declared as a resource.

LegacyAnimation

<LegacyAnimation Name="gNameAnim" Offset="0x1230"/>

Legacy animation header.

PlayerAnimation

<PlayerAnimation Name="gNamePlayerAnim" Offset="0x1230"/>

Player animation header.

PlayerAnimationData

<PlayerAnimationData Name="gNamePlayerAnimData" FrameCount="20" Offset="0x1230"/>

Player animation data.

  • Required attributes: FrameCount

FrameCount is the amount of frames in the animation.