6991fee7f7
Add support for bash autocomplete in main dindexer
...
I'm not sure this is something that should be in the code, since
compgen -W "delete locate navigate query scan" loc
would do just the same as
dindexer --printactions=loc
so this is a duplicated functionality. However this commit is
self-contained, so it should be easy to revert it, if needed.
2016-03-16 00:01:29 +01:00
c6b0fe49b5
Use getopt_long() to parse command line.
2016-03-15 21:10:52 +01:00
96c6e967a0
Cache result of ls command.
2016-03-15 09:33:12 +01:00
d21a4fa374
Use readline
...
Maybe this should be made optional?
2016-03-15 08:42:01 +01:00
0f7a78b395
Fix paths with spaces not being treated correctly.
2016-03-14 19:39:58 +01:00
8fc45ce694
Buildfix on ARM
2016-03-12 00:47:08 +01:00
859764b9a4
Merge branch 'hashdir_refactoring'
2016-03-12 00:37:28 +01:00
e95cd6cc44
Add a --byhash switch to the locate command to search by content.
...
This is mostly to demonstrate the new task classes.
2016-03-12 00:32:15 +01:00
34ead94c8d
Restore DB writing functionality.
...
This ends the scantask refactoring. It is now possible to scan
directories using the new system. Progress feedback is made
available again, although it could use some improvements.
2016-03-10 18:37:42 +01:00
43c8024b0c
Re-add some progress feedback.
...
It's not multithreaded this time o.O
2016-03-10 09:51:27 +01:00
fe7e9c4af9
Turn around Set's tasks so that they provide a SetRecordDataFull.
2016-03-09 21:26:43 +01:00
fd5be6216a
Namespace shortcut
2016-03-09 19:11:45 +01:00
c4c68e3b5c
Take file size in filesearcher.
...
Like this the filesize task I was about to write becomes
unnecessary.
2016-03-09 18:55:03 +01:00
85cbc76de4
Add GeneralFiller helper task to group tasks together.
2016-03-09 09:25:52 +01:00
5798e0d315
New Mime task.
2016-03-09 09:02:17 +01:00
217eac250b
Include only needed headers
2016-03-08 22:56:23 +01:00
c068f93bac
New ContentType task.
...
Fix the problems I encountered while trying to use the new task.
2016-03-08 19:54:04 +01:00
b357bfb2cc
Use basename() instead of the unclear make_rel_path
2016-03-08 19:09:56 +01:00
250b2df846
Add basename() free function.
2016-03-08 19:09:20 +01:00
63c703f20d
Fix wrong hashes for directories
...
set_listing must sort by abs_path, because the path received by
lower_bound() is an absolute path.
Inside hash_dir() the path passed to append_to_vec() is really
the basename of the current item. This will become more clear
in one of the next commits.
2016-03-08 18:58:47 +01:00
7b0189a119
Make debug output a bit nicer.
2016-03-08 08:51:57 +01:00
d2588d3c7e
Implement hashing task.
...
Also get rid of the ShorFileRecordData and put the new LeanBase
class to use.
2016-03-08 08:48:12 +01:00
d579b311f2
Add ShortFileRecordData struct to hold the file list in DirTree.
2016-03-04 21:57:09 +01:00
1956594c01
Move mediatypes.hpp/cpp to machinery.
...
This was needed so that machinery wouldn't depend on common.
Also moved compatibility.h to helpers include dir.
Ideally mediatypes.hpp will be merged into mediatype.hpp, or there
will be some kind of renaming because the two names are
getting really confusing.
2016-03-04 09:01:11 +01:00
6ddf79fad9
Implement MediaType scan task.
2016-03-03 21:31:33 +01:00
5b3e15c45f
Create DirTree class and start moving code around.
2016-03-03 20:46:30 +01:00
b19720548a
Fix program hanging in release.
...
That great idea of spawning a thread for printing to stdout is
firing back. Temporary fix to get past, but the output looks
really broken now.
2016-02-23 23:28:49 +01:00
b55f42e905
Remove debug output.
2016-02-23 21:05:04 +01:00
be9fc3eb0b
Fix various problems introduced in scan with the last commit.
...
Fixes a crash due to an assertion trying to access a
past-the-end iterator.
Gets content type detection to work, although only after
hashing is done (see comment in main.cpp for details).
Fixes a problem with the array passed to guess_content_type
being not sorted as the function expects.
Adds more assertions.
2016-02-23 18:56:07 +01:00
22614432a9
Detect content type after scanning and save to DB.
2016-02-22 19:44:48 +01:00
2e77e4dc0b
Add Glob class and use that one to help guessing content types.
2016-02-20 01:36:31 +01:00
f31463480f
Add stub code to detect VCDs, plus unit test.
2016-02-19 21:57:43 +01:00
7da13f6677
Add assignment operators.
2016-02-19 21:56:10 +01:00
64b5affa4f
Add guess_content_type() function.
...
WiP.
2016-02-19 10:01:11 +01:00
8a97afd6bf
Use Kakoune::SafePtr to hold the PathName in DirIterator.
2016-02-18 21:00:17 +01:00
bb1148863d
Add typedefs to SetListingView<B>
2016-02-17 08:54:38 +01:00
af80771c2e
Fix the new hash_dir().
...
This fixes a bug that I never noticed in the old code.
This also breaks the progress indicator in Release.
2016-02-16 23:01:42 +00:00
5fbad40fda
Add level offset.
...
The level in the FileRecordData is 0 at the base of the
directory being scanned. However PathName::atom_count() can be
more than that. For example scanning /mnt/cdrom will have
level 0 at the mount point, but atom_count() is 2. So it's
necessary to pass in this offset. This way the code can take
that into account and compute stuff correctly.
2016-02-16 20:18:10 +00:00
5908828390
Implement hash_dir() using DirIterator. WiP
...
Keep the old implementation as well, so they can be easily tested.
Uncomment the USE_LEGACY_HASH_DIR define at the top to build with the
old function.
2016-02-16 20:29:50 +01:00
0a3e469951
Support const/non-const DirIterators and SetListingViews
2016-02-16 18:51:17 +01:00
84a2617c24
Enable interoperability between iterators
2016-02-16 18:12:26 +01:00
b94bf308f5
Prepare to allow for non-const iterators.
...
WiP.
2016-02-12 21:02:26 +01:00
bed191c4fc
Better way to construct SetListingView
2016-02-12 20:14:12 +01:00
2a7b8437ed
Make DirIterator forward only
2016-02-12 19:55:32 +01:00
32d32f63cb
Fix SetListing, but some implementation is still missing
2016-02-12 14:44:32 +01:00
5549e7c97d
Show --builtin even when no actions are found.
2016-02-11 09:18:36 +01:00
c89d4b69d5
Add install target.
2016-02-11 09:07:36 +01:00
6362f0f002
Fix unit test
2016-02-10 09:19:54 +01:00
c4a68356f6
Fix a warning in clang
2016-02-10 09:03:31 +01:00
ada0f1df50
Add SetListingView.
...
I'm having linker errors in gtest, pushing to see
if this also happens on the build server.
2016-02-09 23:23:50 +01:00
346946340d
First implementation of set_listing.
...
Trying to move the depth-first iteration logic into
an iterator-like interface. Work in progress WiP
2016-02-09 21:35:13 +01:00
f20d73bdab
Update year on copyright notice.
2016-02-05 21:02:40 +01:00
58c7e917c5
Buildfix on ARM raspberry pi
2016-02-02 20:56:27 +01:00
c5824c36bd
List directories first in the output of ls.
2016-01-29 23:30:59 +00:00
f40600d6e2
Implement ls into subdirectories
2016-01-29 20:15:16 +01:00
362a5d3128
Modernize AutomemBase and add some assertions
2016-01-28 19:02:15 +01:00
394ae7672a
Optimization - replace inner vector with static mem container in dbsource.
2016-01-28 18:13:13 +01:00
8cbd9560ec
Implement listing sets from the db (ls)
2016-01-27 19:18:10 +01:00
9071b25dec
Rename DirManager to GenericPath.
2016-01-26 23:06:36 +01:00
138f112254
Implement cd and pwd commands
2016-01-26 22:53:30 +01:00
c455582a4e
Provide some wrappers so adding commands is more convenient.
2016-01-22 19:54:32 +00:00
99732f6dd8
Add stub for new navigate command.
2016-01-22 15:55:19 +00:00
588d735593
Add missing $3 for the new app_name parameter.
...
Fixes crash after scanning a disk.
2016-01-22 15:54:39 +00:00
eb618f0842
Remove stray cout.
2016-01-22 09:52:34 +00:00
9ee89b815a
Fix mimetype retrieval and make code a bit more readable.
2016-01-22 09:45:20 +00:00
913cdbd59d
Add app_name column to sets.
2016-01-21 20:30:38 +00:00
9f3b27f0b5
Fix scan hanging after listing directories.
2016-01-21 16:56:54 +00:00
e2438f77fd
Fix inserting new files into db.
2016-01-11 14:20:05 +00:00
4c97105d97
Buildfix - pthread is needed when building in Release.
2016-01-11 12:53:37 +00:00
7e737f56a2
Tell libpqtypes to not make copies of strings it receives.
...
The wrapper now copies boost::string_ref into std::string so
it's always safe to pass a string ref, even when it's not
null-terminated.
2016-01-11 12:53:37 +00:00
fa12bb2d74
Bugfix - don't store absolute paths to files in the DB.
...
This was introduced in a previous commit.
2016-01-11 12:53:37 +00:00
3f48d00ad9
Add new str_path_size() method.
...
Refactor code to allow an easier implementation.
2016-01-11 12:53:37 +00:00
5c0a8e0725
Warning fix in Release.
2016-01-07 15:13:03 +00:00
1077661e50
Implement searching for sets by name.
...
Searching for sets also prints the total count of files and
directories in that set.
2016-01-07 15:13:03 +00:00
2593d46ed7
Now there is a cpp for pq_type_helpers.hpp so move code there.
2016-01-07 14:52:32 +00:00
2655ea5f5c
Add support for std::vectors as arrays.
...
Nested vectors are not supported.
2016-01-07 12:57:06 +00:00
f82659e370
Add move semantics.
2016-01-06 13:12:00 +00:00
6edfb08383
Store mime type and charset in the db.
2016-01-06 02:18:42 +00:00
839b9dd49a
Generate a table scheme with some valid username.
...
Have main prog print the new build time constant.
2016-01-05 14:12:36 +00:00
c7b44f6368
Use parametric queries.
2016-01-05 13:30:17 +00:00
f5356e0c29
Change namespace in machinery to mchlib.
2016-01-05 12:49:27 +00:00
4e29200b47
Move scan's core code into new machinery lib.
2016-01-05 12:35:46 +00:00
487b8efe61
Refactor Indexer so that writing to the db is done outside.
2016-01-05 11:24:23 +00:00
c4a2433a04
Move libpqtypes template helpers out to a new header.
2015-12-31 01:20:56 +00:00
ed0fbac5e1
connection.hpp is a public header.
2015-12-31 01:13:26 +00:00
1532d3c083
Fix reading values from ResultSet/Row.
...
Send correct group_id to the db in a safer way.
2015-12-31 00:38:44 +00:00
9bb5689d48
Remove query_void() methods. query() is just fine.
2015-12-30 10:20:52 +00:00
43bec4711e
Clarify PGresult ownership and lifetime.
2015-12-30 10:15:36 +00:00
c6f034d98e
Replace spirit phoenix with standalone phoenix includes.
2015-12-29 19:02:56 +00:00
a3f44bce8d
Use new parametric query_void() instead of manual insert into sets.
2015-12-29 18:11:15 +00:00
390b69e150
Use parametric sql functions to insert new files.
...
Refactoring so that there are no extra copies of data being inserted.
2015-12-29 17:32:22 +00:00
a91e75829f
Fix libpqtypes as external project.
...
Note: due to a bug in cmake/ninja, this change introduces
problems when building with ninja. See:
https://cmake.org/Bug/view.php?id=15256
2015-12-27 03:09:08 +00:00
d8f3578497
Add libpqtypes from http://libpqtypes.esilo.com/
2015-12-26 22:16:51 +00:00
c409a3e0a2
Make main dindexer show options used to build.
2015-12-26 14:51:49 +00:00
05979e24a2
Add split_mime() function.
2015-12-26 14:50:40 +00:00
148b60ee97
Add retrieving the mime type.
...
This is not being stored to the db yet.
2015-12-15 11:26:04 +00:00
2b7d8a6783
Don't scatter the casted type throughout the code.
2015-12-14 13:05:13 +00:00
45e5b7bc8d
Implement "locate" action.
2015-12-14 13:00:10 +00:00
cc5bb9bc34
Sort commands printed in help output.
...
Also fix const-correctness. Hopefully.
2015-12-14 11:55:22 +00:00
57d925034d
Begin working on the "query" action.
2015-12-13 16:46:42 +00:00
93a5130cc5
Move ValidationError to commonlib.
2015-12-13 16:33:58 +00:00
986f0c8db3
Buildfix and minor changes
2015-12-13 16:14:05 +00:00
bef901c56c
Print git revision in --version.
2015-12-12 20:27:31 +00:00
861ea2c809
Implement deleting sets from the db.
2015-12-12 20:20:57 +00:00
4a0ddb4beb
Hide expand() implementation into settings.cpp.
2015-12-12 20:02:42 +00:00
74afa9f502
Rename settings stuff as it got moved to commonlib.
2015-12-12 17:41:34 +00:00
6e7176be10
Move DBSettings to common lib.
2015-12-12 17:23:50 +00:00
8a4afa241e
Begin work on the new delete action.
2015-12-12 16:53:09 +00:00
10b9be1f85
Print human-readable autodetected media type.
2015-12-10 16:31:11 +00:00
6c30621400
Put some common stuff into a separate library.
2015-12-10 15:33:38 +00:00
a0b87e6a2d
Manage read errors.
...
If instructed to continue on errors, store info correctly in the db.
2015-12-10 12:13:16 +00:00
ed3dea8f2c
More assertions and more explicit code.
2015-12-09 18:36:46 +00:00
3a4c11ad4c
Buildfix when blkid.h is not available.
2015-12-08 16:13:36 +00:00
2cdf0cb3b9
Allow disabling media autodetection at build time.
...
Use DINDEXER_WITH_MEDIA_AUTODETECT.
2015-12-08 16:07:21 +00:00
97de157724
Refactor the new media type autodetection code.
2015-12-08 16:03:30 +00:00
c82b41207e
Enable more warnings (and fix some).
2015-12-08 13:57:12 +00:00
fe2ea40c4f
Implement autodetect set type.
...
Only supports HardDisk, Directory, CdRom, DVD, BluRay for now.
2015-12-08 13:56:46 +00:00
87bc031e65
Rename update action to scan.
2015-12-04 19:40:02 +00:00
20ebc10b0c
Dynamically build the type list for the help message.
2015-12-04 19:32:57 +00:00
8708ef9503
Add dispatcher program.
...
Invoke like dindexer <action> a b c and it will exec
ACTIONS_PATH/dindexer-<action> a b c
or
ACTIONS_PATH/<action>/dindexer-<action> a b c
2015-12-04 16:32:32 +00:00
fca9b94afd
Make one more empty target that only carries include paths.
2015-12-04 16:30:41 +00:00
3899680544
Pull lengthof definition into a different header.
2015-12-04 13:48:20 +00:00
d8be48d2e8
Reorganize source code in directories.
2015-12-04 12:58:13 +00:00
3c1b7108f3
Allow type "C" (for CD-Rom)
2015-12-04 12:30:48 +00:00
5f8756ef1b
Store access time and modify time in the db.
2015-12-01 16:35:46 +00:00
f1c3deea63
Group parameters in a struct when calling filesearcher's callback.
2015-12-01 16:34:22 +00:00
8b9241757d
Don't save to DB if the disk has been added already.
2015-11-27 20:41:36 +00:00
e957fde12c
Enable uppercase hash-to-string conversion.
2015-11-27 20:34:42 +00:00
51d0bfdf7a
Avoid messages clashing on stdout in debug.
...
This quickfix completely disables progress output in debug.
2015-11-27 19:54:03 +00:00
f6c638e6e4
Fix wrong assertion.
2015-11-27 19:49:29 +00:00
56735c9d86
Get the config file path from cmake.
2015-11-13 19:25:32 +00:00
f48626960d
Improve progress output.
2015-11-13 15:45:28 +00:00
f7441292bc
Add optional verbose mode that shows the number of hashed files.
2015-11-12 14:07:26 +00:00
a82ab4a4ed
Bugfix in Release.
2015-11-11 20:13:03 +00:00
49a9702115
Make a macro for verbosity.
2015-11-11 20:12:55 +00:00
911da3bb00
Add column 'type' to sets and take its value on command line.
2015-11-11 20:06:14 +00:00
585c7f45b7
Allow passing set name and search path as command line parameters.
2015-11-11 19:24:10 +00:00
9a25846880
Fix release build.
2015-11-11 17:14:51 +00:00
292e32b38f
Read db connection parameters from a yml file.
2015-11-11 17:08:17 +00:00
413bf0444b
Update DB tables and insert set along with new files.
2015-11-11 13:30:41 +00:00
444d2a2fa3
Limit insertion to 100 records per query, but this is still suboptimal.
2015-11-11 12:35:52 +00:00
4236b2ece8
Fix hashing and db insertion.
2015-11-11 12:28:04 +00:00
bd9ce0ef54
Send data to the database.
2015-11-11 02:01:37 +00:00
a00d30b0ee
Calculate hash for all entries.
2015-11-10 17:48:22 +00:00
1dee8e0f83
Implement SSE2 Tiger hashing.
2015-11-09 16:50:56 +00:00
d07dc9a7da
New Indexer class.
2015-11-06 19:48:38 +00:00
4fcc2cf25c
New make_relative_path() function.
2015-11-06 19:48:18 +00:00
3f8cda06aa
Improve interface and add some getters.
2015-11-06 19:48:02 +00:00
485796db48
Keep paths in a string pool.
2015-11-06 17:57:07 +00:00
109ce9b82a
Use string_ref instead of string and implement join().
2015-11-05 17:22:34 +01:00
31a70945bb
First import
2015-11-05 15:32:31 +01:00
