orotool/README.md

# orotool #

Your Origins RO API client

## Purpose ##

Work in progress

## Licence ##

orotool is released under the GPL3+

## Compiling the project ##

Work in progress

### Dependencies ###

* [lzma](https://tukaani.org/xz/) (optional)
* [curlcpp](https://github.com/JosephP91/curlcpp) (compulsory at the moment, bundled)
* curl (compulsory at the moment)
* libev (compulsory)
* [boost](https://www.boost.org/) (compulsory)
* [simdjson](https://github.com/simdjson/simdjson) (compulsory at the moment, bundled)
* [restc-cpp](https://github.com/jgaa/restc-cpp) (optional, bundled, currently broken)
* [RapidJSON](https://rapidjson.org/) (optional, currently broken)
* [SQLiteCpp](https://srombauts.github.io/SQLiteCpp/) (compulsory at the moment, bundled)
* SQLite (compulsory at the moment)
* [date](https://howardhinnant.github.io/date/date.html) (compulsory, bundled)
* [duckhandy](https://alarmpi.no-ip.org/gitan//King_DuckZ/duckhandy/) (compulsory, bundled)
* [sprout](https://github.com/bolero-MURAKAMI/Sprout) (compulsory)
* [magic_enum](https://github.com/Neargye/magic_enum/) (compulsory, bundled)
* [gnulib](https://git.savannah.gnu.org/gitweb/?p=gnulib.git) (compulsory, bundled)

## Contact ##

The best way to get in touch is on IRC:

* *Rizon* - irc.rizon.net:6697 - `King_DuckZ` - #OriginsRO
* *Freenode* - chat.freenode.net:6697 - `King_DuckZ`

Other ways you can get in touch:

* Matrix: `@duckz:alarmpi.no-ip.org`
* XMPP: `king_duckz@jabber.at`
* Discord: `DuckZ#0896`

## Recipies ##

This is a collection of tricks, commands and SQL queries that you might find helpful.

### Exporting JSON to file ###

If you saved JSON data to the database in compressed format (`store_raw_json=true` and `json_store_mode=xz`) you can export them to separate files with the following bash command (assuming current directory contains a originsro.db3 DB):

```
mkdir -pv split_json
cd split_json

digits=0000; n=0; while IFS= read -r line; do \
base64 --decode <<<"$line" | unxz --decompress > output"${digits:${#n}:${#digits}}${n}".txt; \
(( n++ )); \
done < <(sqlite3 ../debug/originsro.db3 'select source from source_store where format=2')
```

This will not export plain uncompressed JSON (`format=1`)

### Getting min/max price of each item on the market now ###

With this query you can retrieve the min and max price of each item currently being sold (assuming database is up to date), along with their names, the name of the shop selling them and the total count of items of that type across all shops and all prices. Note that refined equipment or equipment with cards are considered to be all equivalent to the plain item.

```
SELECT title, name, MIN(price) AS min_price, MAX(price) AS max_price, SUM(amount) AS amount FROM (
SELECT shops.title, items.name, shop_items.price, items.item_id,
	shop_items.amount
	FROM shops
	LEFT JOIN shop_snapshots ON shop_snapshots.shop_id=shops.id
	LEFT JOIN shop_items ON shop_items.snapshot_id=shop_snapshots.id
	LEFT JOIN items ON shop_items.item_id=items.id
	WHERE shops.type=1 AND
		shop_snapshots.last_seen_date=(SELECT max(last_seen_date) FROM shop_snapshots)
) GROUP BY item_id
```

"Currently open" means shops seen on the same date the most recent shop was seen. In my current case my database was last updated 6 hours ago, this means I'm seeing the list of items that were being sold 6 hours ago.

### Estimate JSON storage size ###

This returns an estimate in MiB of the total storage occupied by the saved JSON data

```
SELECT ROUND(CAST(SUM(pgsize) AS REAL)/(1024.0*1024.0), 2) FROM dbstat WHERE name='source_store'
```

### Deleting JSON logs ###

You can delete the stored JSON whenever you want. It is adviced that you also set all `source_id` references to 0 in order to avoid wrong references to the new JSON logs getting stored in the future:

```
UPDATE items SET source_id=0;
UPDATE fame_list SET source_id=0;
UPDATE icons SET source_id=0;
UPDATE shop_snapshots SET source_id=0;
DROP TABLE IF EXISTS source_store;
VACUUM;
```