orotool

Your Origins RO API client

Purpose

Work in progress

Licence

orotool is released under the GPL3+

Compiling the project

Work in progress

Dependencies

• lzma (optional)
• curlcpp (compulsory at the moment, bundled)
• curl (compulsory at the moment)
• libev (compulsory)
• boost (compulsory)
• simdjson (compulsory at the moment, bundled)
• restc-cpp (optional, bundled, currently broken)
• RapidJSON (optional, currently broken)
• SQLiteCpp (compulsory at the moment, bundled)
• SQLite (compulsory at the moment)
• date (compulsory, bundled)
• duckhandy (compulsory, bundled)
• sprout (compulsory)
• magic_enum (compulsory, bundled)
• gnulib (compulsory, bundled)

Contact

The best way to get in touch is on IRC:

• Rizon - irc.rizon.net:6697 - King_DuckZ - #OriginsRO
• Freenode - chat.freenode.net:6697 - King_DuckZ

Other ways you can get in touch:

• Matrix: @duckz:alarmpi.no-ip.org
• XMPP: king_duckz@jabber.at
• Discord: DuckZ#0896

Recipes

This is a collection of tricks, commands and SQL queries that you might find helpful.

Exporting JSON to file

If you saved JSON data to the database in compressed format (store_raw_json=true and json_store_mode=xz) you can export them to separate files with the following bash command (assuming current directory contains a originsro.db3 DB):

mkdir -pv split_json
cd split_json

digits=0000; n=0; while IFS= read -r line; do \
base64 --decode <<<"$line" | unxz --decompress > output"${digits:${#n}:${#digits}}${n}".txt; \
(( n++ )); \
done < <(sqlite3 ../debug/originsro.db3 'select source from source_store where format=2')

This will not export plain uncompressed JSON (format=1)

Getting min/max price of each item on the market now

With this query you can retrieve the min and max price of each item currently being sold (assuming database is up to date), along with their names, the name of the shop selling them and the total count of items of that type across all shops and all prices. Note that refined equipment or equipment with cards are considered to be all equivalent to the plain item.

SELECT title, name, MIN(price) AS min_price, MAX(price) AS max_price, SUM(amount) AS amount FROM (
SELECT shops.title, items.name, shop_items.price, items.item_id,
	shop_items.amount
	FROM shops
	LEFT JOIN shop_snapshots ON shop_snapshots.shop_id=shops.id
	LEFT JOIN shop_items ON shop_items.snapshot_id=shop_snapshots.id
	LEFT JOIN items ON shop_items.item_id=items.id
	WHERE shops.type=1 AND
		shop_snapshots.last_seen_date=(SELECT max(last_seen_date) FROM shop_snapshots)
) GROUP BY item_id

"Currently open" means shops seen on the same date the most recent shop was seen. In my current case my database was last updated 6 hours ago, this means I'm seeing the list of items that were being sold 6 hours ago.

Estimate JSON storage size

This returns an estimate in MiB of the total storage occupied by the saved JSON data

SELECT ROUND(CAST(SUM(pgsize) AS REAL)/(1024.0*1024.0), 2) FROM dbstat WHERE name='source_store'

Deleting JSON logs

You can delete the stored JSON whenever you want. It is adviced that you also set all source_id references to 0 in order to avoid wrong references to the new JSON logs getting stored in the future:

UPDATE items SET source_id=0;
UPDATE fame_list SET source_id=0;
UPDATE icons SET source_id=0;
UPDATE shop_snapshots SET source_id=0;
DROP TABLE IF EXISTS source_store;
VACUUM;