mirror of
https://github.com/zeldaret/oot.git
synced 2025-08-08 07:20:16 +00:00
New assets system (#2481)
* wip: New assets system tm
Builds gc-eu-mq-dbg OK from clean after
1) make setup
2) python3 -m tools.assets.extract -j
3) replace 0x80A8E610 with sShadowTex in extracted/gc-eu-mq-dbg/assets/overlays/ovl_En_Jsjutan/sShadowMaterialDL.inc.c
4) make various symbols in extracted data like sTex static
* use variables from config.yml for gMtxClear and sShadowTex addresses
* Write source with static for overlays using `HACK_IS_STATIC_ON` hack
* gc-eu-mq-dbg OK from clean with `make setup && make`
* implement more skeleton-related types, cleanups, fixups
* fix extracted data to no longer produce compilation warnings
* implement more of RoomShapeImage types
* yeet XmlPath from ExternalFile usage
* Implement PlayerAnimationDataResource (link_animetion data)
* fix csdis CS_TIME extra arg
* dmadata file names no longer hardcoded for gc-eu-mq-dbg
* ntsc-1.0 OK
* xml fixes
* slightly improve standard output
* rm extract_assets.py
* generate and use Limb enums (TODO: check Skin skels and implement for Curve skels)
* handle dependencies between xmls
* introduce RawPointers xml attribute to ignore specific pointers and keep them raw
* add tools/extract_assets.sh
* fixups
* only extract if xmls changed or if -f (force) is used
* fixups, gc-eu OK
* all versions OK
* check attributes of xml resources elements
* Implement legacy skelanime resources
* fix ASSET_FILES_BIN_EXTRACTED/COMMITTED: look for .u8.bin specifically instead of just .bin
* implement JFIFResource
* fix png/jpg wildcards: look specifically for .u64.png .u32.png .u64.jpg
* Makefile: Add rules to build .png, .bin and .jpg in assets/ too
* start writing actual docs
* extract sTransCircleDL and sTransWipeDL
* misc cleanup/fixes, pygfxd 1.0.3
* refactor CDataExt.set_write callback args to use a dataclass
* Move {} to in-source
* misc
* more progress on spec
* fix missing braces in n64dd_error_textures.c
* finish xml spec doc
* assets xmls fixes
* some cleanup, use `gNameTex_WIDTH/HEIGHT` macros in dlists
* handle hackmode_syotes_room, fix compile
* C build_from_png
* rm tools/assets/bin2c
* rm ZAPD
* format
* remove rule to generate dmadata_table.py
* CC0 license (and some import cleanup)
* dont try to build zapd (rmd)
* simplify palettes with single user (ci images with a non-shared palette)
* add docs on how images are handled
* bss
* allow -j N
* fix n64texconv python bindings memory management
* move -j at the end of calling extraction script
* with -j, update last_extracts.json as each job completes rather than only if all complete
* make interrupting less jank by making child processes ignore sigint
* use enum names in `SCENE_CMD_SKYBOX_SETTINGS`
* `multiprocessing.get_context("fork")`
* import rich, except ImportError s
* fix optional rich usage
* .bss
* .bss
* .bss
* assets extraction: -j -> -j$(N_THREADS)
* .bss
* change LIMB_NONE/MAX defaults to be FILE_OFFSET instead of SKELNAME
* 0XHEX -> 0xHEX
* fix bss
* Proper includes for assets
mostly proper, some includes like dlists resources always causing a sys_matrix.h include (when not every dlist references gIdentityMtx) could be done better
* rm z64.h
* rm z64.h take two
* bss
* Make .u64 suffix for pngs optional
* fixup: rm .u64 suffix from n64dd image paths
* Remove elemtype suffixes from .bin and .jpg files
* Update images.md
* some build_from_png cleanup, more error handling, comments
* Handle skybox textures
Introduce "sub-format" suffix for pngs, with sub-formats split_lo and split_hi being used for skybox textures
* fixup for older python
* improve collision output some
* fully use SURFACETYPE[01] macros in writing extracted surface types
* use WATERBOX_PROPERTIES in extracted waterboxes
* some SceneCommandsResource cleanup
* format EnvLightSettingsList output
This commit is contained in:
Dragorn421
GitHub
parent
0c6c112cb9
commit
1e556e3a3d
460 changed files with 14342 additions and 48656 deletions
291
tools/assets/descriptor/spec.md
Normal file
291
tools/assets/descriptor/spec.md
Normal file
|
|
@ -0,0 +1,291 @@
|
|||
This document describes the expected structure of xml files describing assets.
|
||||
|
||||
# Top elements
|
||||
|
||||
## `Root`
|
||||
|
||||
```xml
|
||||
<Root>
|
||||
...
|
||||
</Root>
|
||||
```
|
||||
|
||||
This is the root element in the file, containing exclusively `<File>` and `<ExternalFile>` elements as direct children.
|
||||
|
||||
## `File`
|
||||
|
||||
```xml
|
||||
<File Name="baserom_file" Segment="11">
|
||||
...
|
||||
</File>
|
||||
```
|
||||
|
||||
A `<File>` contains resources elements as children.
|
||||
|
||||
- Required attributes: `Name`
|
||||
- Optional attributes: `Segment`
|
||||
|
||||
`Name` is the name of the baserom file from which the data is to be extracted.
|
||||
|
||||
`Segment` (decimal) is the segment number for the file.
|
||||
|
||||
## `ExternalFile`
|
||||
|
||||
```xml
|
||||
<ExternalFile OutPath="assets/name"/>
|
||||
```
|
||||
|
||||
Declare the `<File>`s in the xml may reference symbols from an external file.
|
||||
|
||||
The external file is located by matching its name against the list of assets in the version's `config.yml`.
|
||||
|
||||
For example, `baseroms/gc-eu/config.yml` contains
|
||||
|
||||
```yml
|
||||
assets:
|
||||
- name: objects/gameplay_keep
|
||||
xml_path: assets/xml/objects/gameplay_keep_pal.xml
|
||||
```
|
||||
|
||||
then `<ExternalFile OutPath="assets/objects/gameplay_keep/"/>` refers to that gameplay_keep entry, which uses the `gameplay_keep_pal.xml` xml file when extracting assets for version gc-eu.
|
||||
|
||||
|
||||
# Resource elements
|
||||
|
||||
Resource elements describe resources. Resources are pieces of data corresponding to a symbol each.
|
||||
|
||||
Two attributes are required on all resource elements: `Name` and `Offset`.
|
||||
|
||||
- `Name` is the name of the symbol associated to the resource.
|
||||
- `Offset` is the location in bytes from the start of the file data.
|
||||
|
||||
## `Blob`
|
||||
|
||||
```xml
|
||||
<Blob Name="gNameBlob" Size="0x421" Offset="0x1230">
|
||||
```
|
||||
|
||||
Unstructured binary data.
|
||||
|
||||
- Required attributes: `Size`
|
||||
|
||||
`Size` is the size of the binary blob in bytes.
|
||||
|
||||
## `DList`
|
||||
|
||||
```xml
|
||||
<DList Name="gNameDL" Offset="0x1230" Ucode="f3dex2" RawPointers="0x08000000,0x09000000"/>
|
||||
```
|
||||
|
||||
A display list.
|
||||
|
||||
- Optional attributes: `Ucode`, `RawPointers`
|
||||
|
||||
`Ucode` (defaults to `f3dex2`) picks the graphics microcode for which to disassemble the dlist. It may be `f3dex` or `f3dex2`.
|
||||
|
||||
`RawPointers` (defaults to an empty value) is a comma-separated list of values the display list uses as raw pointers ("hex" instead of a symbol). The purpose of this attribute is to silence extraction warnings.
|
||||
|
||||
## `Mtx`
|
||||
|
||||
```xml
|
||||
<Mtx Name="gNameMtx" Offset="0x1230"/>
|
||||
```
|
||||
|
||||
A fixed-point matrix.
|
||||
|
||||
## `Texture`
|
||||
|
||||
```xml
|
||||
<Texture Name="gNameTex" Format="rgba16" Width="16" Height="16" Offset="0x1230"/>
|
||||
<Texture Name="gNameTex" Format="ci8" Width="16" Height="16" Offset="0x1230" TlutOffset="0x2340"/>
|
||||
<Texture Name="gNameTex" Format="ci8" Width="16" Height="16" Offset="0x1230" ExternalTlut="baserom_file" ExternalTlutOffset="0x2340"/>
|
||||
```
|
||||
|
||||
A texture, an image in one of the native N64 formats.
|
||||
|
||||
- Required attributes for all formats: `Format`, `Width`, `Height`
|
||||
- Required attributes for CI formats (`ci4`, `ci8`): `TlutOffset`, or `ExternalTlut` and `ExternalTlutOffset`
|
||||
|
||||
`Format` is the format of the texture, one of `rgba32`, `rgba16`, `i4`, `i8`, `ia4`, `ia8`, `ia16`, `ci4` or `ci8`.
|
||||
|
||||
`Width` and `Height` specify the dimensions of the texture.
|
||||
|
||||
For CI formats, the TLUT (Texture Look Up Table, or palette) must be specified with either `TlutOffset` if the TLUT is in the same file as the texture, or both of `ExternalTlut` and `ExternalTlutOffset` if the TLUT is in a different file. `ExternalTlut` is the name of the baserom file where the TLUT is. In both cases, the TLUT must also be declared as a resource.
|
||||
|
||||
## `Array`
|
||||
|
||||
```xml
|
||||
<Array Name="gNameVtx" Count="42" Offset="0x1230">
|
||||
<Vtx/>
|
||||
</Array>
|
||||
<Array Name="gNameVec3sArray" Count="42" Offset="0x1230">
|
||||
<Vector Type="s16" Dimensions="3"/>
|
||||
</Array>
|
||||
<Array Name="gNameS16Array" Count="42" Offset="0x1230">
|
||||
<Scalar Type="s16"/>
|
||||
</Array>
|
||||
```
|
||||
|
||||
An array of vertices, vectors or scalars. The child element determines the array's element type.
|
||||
|
||||
- Required attributes: `Count`
|
||||
|
||||
`Count` is the length of the array.
|
||||
|
||||
The child element may be one of `<Vtx/>` (for `Vtx[]`), `<Vector Type="s16" Dimensions="3"/>` (for `Vec3s[]`) or `<Scalar Type="s16">` (for `s16[]`).
|
||||
|
||||
## `Scene`
|
||||
|
||||
```xml
|
||||
<Scene Name="name" Offset="0x1230"/>
|
||||
```
|
||||
|
||||
Scene commands.
|
||||
|
||||
## `Room`
|
||||
|
||||
```xml
|
||||
<Room Name="name" Offset="0x1230"/>
|
||||
```
|
||||
|
||||
Room commands.
|
||||
|
||||
## `Collision`
|
||||
|
||||
```xml
|
||||
<Collision Name="gNameCol" Offset="0x1230"/>
|
||||
```
|
||||
|
||||
Collision header.
|
||||
|
||||
## `Cutscene`
|
||||
|
||||
```xml
|
||||
<Cutscene Name="gNameCs" Offset="0x1230"/>
|
||||
```
|
||||
|
||||
Cutscene script.
|
||||
|
||||
## `Path`
|
||||
|
||||
```xml
|
||||
<Path Name="gNamePathList" Offset="0x1230" NumPaths="12"/>
|
||||
```
|
||||
|
||||
Path list.
|
||||
|
||||
- Required attributes: `NumPaths`
|
||||
|
||||
`NumPaths` is the length of the path list.
|
||||
|
||||
## `Skeleton`
|
||||
|
||||
```xml
|
||||
<Skeleton Name="gNameSkel" Type="Normal" LimbType="Standard" LimbNone="NAME_LIMB_NONE" LimbMax="NAME_LIMB_MAX" EnumName="NameLimb" Offset="0x1230"/>
|
||||
```
|
||||
|
||||
Skeleton header.
|
||||
|
||||
- Required attributes: `Type`, `LimbType`
|
||||
- Optional attributes: `LimbNone`, `LimbMax`, `EnumName`
|
||||
|
||||
`Type` is the type of the skeleton, one of `Normal`, `Flex` or `Curve`.
|
||||
|
||||
`LimbType` is the type of limb used in the skeleton, one of `Standard`, `LOD`, `Skin`, `Curve` or `Legacy`.
|
||||
|
||||
Not all skeleton types are compatible with all limb types:
|
||||
|
||||
`LimbType` | Compatible skeleton `Type`
|
||||
-----------|---------------------------
|
||||
`Standard` | `Normal`, `Flex`
|
||||
`LOD` | `Normal`, `Flex`
|
||||
`Skin` | `Normal`
|
||||
`Curve` | `Curve`
|
||||
`Legacy` | none
|
||||
|
||||
`LimbNone`, `LimbMax`, `EnumName` can be set to override the corresponding names in the generated limb enum:
|
||||
|
||||
```c
|
||||
typedef enum NameLimb {
|
||||
NAME_LIMB_NONE,
|
||||
...
|
||||
NAME_LIMB_MAX
|
||||
} NameLimb;
|
||||
```
|
||||
|
||||
## `LimbTable`
|
||||
|
||||
```xml
|
||||
<LimbTable Name="gNameLimbs" LimbType="Standard" Count="12" Offset="0x1230"/>
|
||||
```
|
||||
|
||||
Limb table.
|
||||
|
||||
- Required attributes: `LimbType`, `Count`
|
||||
|
||||
`LimbType` is one of `Standard`, `LOD`, `Skin`, `Curve` or `Legacy`.
|
||||
|
||||
`Count` is the amount of limbs.
|
||||
|
||||
## `Limb`
|
||||
|
||||
```xml
|
||||
<Limb Name="gNameLimb" LimbType="Standard" EnumName="NAME_LIMB_NAME" Offset="0x1230"/>
|
||||
```
|
||||
|
||||
Limb of a skeleton.
|
||||
|
||||
- Required attributes: `LimbType`
|
||||
- Optional attributes: `EnumName`
|
||||
|
||||
`LimbType` is one of `Standard`, `LOD`, `Skin`, `Curve` or `Legacy`.
|
||||
|
||||
`EnumName` can be set to override the limb name in the generated limb enum.
|
||||
|
||||
## `Animation`
|
||||
|
||||
```xml
|
||||
<Animation Name="gNameAnim" Offset="0x1230"/>
|
||||
```
|
||||
|
||||
Animation header.
|
||||
|
||||
## `CurveAnimation`
|
||||
|
||||
```xml
|
||||
<CurveAnimation Name="gNameAnim" SkelOffset="0x120" Offset="0x1230"/>
|
||||
```
|
||||
|
||||
Curve animation header.
|
||||
|
||||
- Required attributes: `SkelOffset`
|
||||
|
||||
`SkelOffset` is the offset of the skeleton which uses this animation. The skeleton must also be declared as a resource.
|
||||
|
||||
## `LegacyAnimation`
|
||||
|
||||
```xml
|
||||
<LegacyAnimation Name="gNameAnim" Offset="0x1230"/>
|
||||
```
|
||||
|
||||
Legacy animation header.
|
||||
|
||||
## `PlayerAnimation`
|
||||
|
||||
```xml
|
||||
<PlayerAnimation Name="gNamePlayerAnim" Offset="0x1230"/>
|
||||
```
|
||||
|
||||
Player animation header.
|
||||
|
||||
## `PlayerAnimationData`
|
||||
|
||||
```xml
|
||||
<PlayerAnimationData Name="gNamePlayerAnimData" FrameCount="20" Offset="0x1230"/>
|
||||
```
|
||||
|
||||
Player animation data.
|
||||
|
||||
- Required attributes: `FrameCount`
|
||||
|
||||
`FrameCount` is the amount of frames in the animation.
|
||||
Loading…
Add table
Add a link
Reference in a new issue