Commit graph

Select branches

Hide pull requests

dev

master

scraplang

arabica

Mono Color

• fa08abd00d Use iconv for converting html *before* passing input to tidyhtml. scraplang King_DuckZ 2020-04-02 19:50:28 +02:00
• d64a4af105 Bugfix - clone last entry, not first King_DuckZ 2020-04-02 19:47:28 +02:00
• 9cd2608406 Allow comments in scraplang. King_DuckZ 2020-04-02 16:42:35 +02:00
• 5a9e4e09a4 Remove the DOCTYPE comment some html pages have since it cause xqilla to freeze King_DuckZ 2020-04-02 16:12:37 +02:00
• 329ccef6ef Slightly improve code. King_DuckZ 2020-04-02 02:28:30 +02:00
• 4958a83ddb Trying to get default value filling fixed. King_DuckZ 2020-04-02 02:26:44 +02:00
• 830ab42c49 Remove useless namespace thing King_DuckZ 2020-04-01 17:32:12 +02:00
• 32f87e5185 Add using std::string to reduce clutter. King_DuckZ 2020-04-01 03:16:11 +02:00
• b79d758e8e Fix error when running multiple xpaths or something like that. King_DuckZ 2020-04-01 03:14:26 +02:00
• b536026f58 Setting a default namespace breaks queries when namespace is empty, so make it a parameter. King_DuckZ 2020-04-01 03:09:45 +02:00
• 55eb7c1fc0 XQUERY3 seems to work, gow for it!! King_DuckZ 2020-03-31 20:11:30 +02:00
• bdb858de5a Ignore the stupid http://www.w3.org/1999/xhtml namespace some html have King_DuckZ 2020-03-31 20:11:03 +02:00
• 6e35c880a4 Ask server to gzip data. King_DuckZ 2020-02-19 17:22:08 +01:00
• 33866b3d6b Add --from-code option for users to force the source charset. King_DuckZ 2020-02-19 17:21:20 +01:00
• 54ac44b81d Remove --dump-raw option. King_DuckZ 2020-02-19 17:13:34 +01:00
• 3dcbd48067 Fix hardcoded "always read from stdin" problem King_DuckZ 2020-02-18 14:58:25 +01:00
• 5de2dfbe70 Allow empty default string values King_DuckZ 2020-02-18 11:53:04 +01:00
• d97cf03a34 Fix some exceptions crashing the program King_DuckZ 2020-02-18 11:52:44 +01:00
• 7170347969 Update submodules King_DuckZ 2020-02-18 10:43:07 +01:00
• 60d6c2cb61 Working on better scraplang support, still not there tho. King_DuckZ 2020-02-18 10:27:52 +01:00
• 430886085c Use XQilla and Xerces-c from the system instead of pugixml. King_DuckZ 2020-02-18 10:19:51 +01:00
• 9dba8043f1 Set more options on tidyhtml. King_DuckZ 2018-02-19 20:11:17 +00:00
• 494364c22e Put the always-stdin symbol in the right place. King_DuckZ 2018-02-15 10:39:46 +00:00
• 5d2c5863a5 Making ApplyBlocks work with {{variable}} sources. King_DuckZ 2018-02-15 10:29:05 +00:00
• b028e8c492 Lots of crap but it works. I'll improve code as I go. King_DuckZ 2018-02-08 00:57:16 +00:00
• a6916f6179 Read from stdin when source is - King_DuckZ 2018-02-08 00:56:04 +00:00
• 1d750ad2f9 Accept - as a valid source URL. King_DuckZ 2018-02-08 00:54:33 +00:00
• 76f403b3ce Extract read_all() functions into a separate file. King_DuckZ 2018-02-08 00:54:17 +00:00
• 84a599e771 Fix the code so it builds & runs. King_DuckZ 2018-02-05 21:41:38 +00:00
• 79ac7534f2 markdown code formatting King_DuckZ 2018-02-05 21:40:36 +00:00
• a9ff092401 WiP - do item counting in mstch variants correctly. King_DuckZ 2018-01-30 10:39:33 +00:00
• 8d2c9f9013 Update tidy and curlcpp submodules. King_DuckZ 2018-01-18 14:06:27 +00:00
• b39621ea51 Fix the build but the code is still untested. King_DuckZ 2018-01-18 00:16:17 +00:00
• 6dffe9b848 Writing the code to go from tree to mustache dictionary. King_DuckZ 2018-01-17 23:24:35 +00:00
• 41bb315b02 Store the same tree for ApplyBlocks too. King_DuckZ 2018-01-17 01:29:46 +00:00
• 2fd4daf52c Keep FromBlock data in tree form. King_DuckZ 2018-01-16 19:50:35 +00:00
• 26b912d66c Allow dots in scraplang identifiers. King_DuckZ 2018-01-16 10:42:25 +00:00
• 3572803f66 Allow nesting of structs. King_DuckZ 2018-01-16 10:23:15 +00:00
• fcb25ed456 WiP reworking the AST interpreter. King_DuckZ 2018-01-13 18:16:11 +00:00
• 29f8fe299e Still trying to get the AST interpreted without much luck. King_DuckZ 2018-01-13 02:03:01 +00:00
• f0e7a1d136 Trying to get scraplang implemented King_DuckZ 2018-01-10 11:09:56 +00:00
• c31d317d51 Print a notice in the --help view in debug builds. master King_DuckZ 2015-10-01 15:37:09 +02:00
• 41b0f59039 Bump version to 0.2.1b King_DuckZ 2015-10-01 15:32:30 +02:00
• 5d0a895978 Attach GPLv3. King_DuckZ 2015-10-01 15:31:58 +02:00
• bdd50d2267 Refactor xpath query into a separate function. King_DuckZ 2015-10-01 14:18:02 +02:00
• c9db1d8ba3 Wrap the unique_ptr so that dtor is called from the cpp. King_DuckZ 2015-10-01 01:45:42 +02:00
• dfd0ec343e Implement parsing of scraplang. King_DuckZ 2015-10-01 01:32:27 +02:00
• 7dfd1f4a38 Update tidy (fixes the build with tidy as a submodule) King_DuckZ 2015-09-30 16:33:52 +02:00
• bf3b85498b Add an option to customize the user agent at runtime. King_DuckZ 2015-09-30 01:27:28 +02:00
• c947eab83f Show some readable message when being passed an unknown option. King_DuckZ 2015-09-30 01:14:47 +02:00
• 05af365c58 Move command line parsing code to a new file. King_DuckZ 2015-09-30 01:13:48 +02:00
• c304ffbbf0 Don't detect if it's a tty - only read from stdin when url is - King_DuckZ 2015-09-29 21:01:33 +02:00
• db1311839d Check fstats instead of using isatty(). dev King_DuckZ 2015-09-29 17:40:01 +02:00
• c69252604c Default to static tidy-html5, but let the user configure this. King_DuckZ 2015-09-28 23:44:11 +02:00
• 49aa62815a Allow piping. King_DuckZ 2015-09-28 23:37:42 +02:00
• 943e760ffd Add dump parameters. King_DuckZ 2015-09-28 23:24:23 +02:00
• 00150938dd Fix the html cleaning code that was not really cleaning. King_DuckZ 2015-09-28 22:59:09 +02:00
• 8e517e5de9 Parse options through boost program_options. King_DuckZ 2015-09-28 21:48:46 +02:00
• 4f85fa01a9 Update libtidy and curlcpp. King_DuckZ 2015-09-28 15:30:09 +02:00
• 44992458ac Quick dirty fix to avoid invalid characters in scripts. King_DuckZ 2015-03-01 05:03:12 +01:00
• 3bfea89568 Drop tidy from the repo and import it as submodule. King_DuckZ 2015-03-01 03:17:47 +01:00
• 0e077a4930 Refactoring to put html retrieval & cleaning into a separate file. King_DuckZ 2014-06-07 22:07:13 +02:00
• cb00e484fa Working example. King_DuckZ 2014-06-07 20:44:43 +02:00
• aa015ddd6a Working example. King_DuckZ 2014-06-07 01:15:06 +02:00
• c9de3d3389 Updating to tidy-html5. King_DuckZ 2014-06-06 22:33:27 +02:00
• e2d74fd092 Trying to use libtidy but it throws. King_DuckZ 2014-06-06 22:22:12 +02:00
• 56f0736d1a Move headers into tidy/ subdirectory. King_DuckZ 2014-06-06 21:34:01 +02:00
• 3182e098bb Import of libtidy with custom cmake file. King_DuckZ 2014-06-06 21:18:30 +02:00
• 3613b106bc Try using Arabica. arabica King_DuckZ 2014-06-06 20:26:14 +02:00
• f213ce5411 First import King_DuckZ 2014-06-06 20:24:24 +02:00