45ecdb9a2b
Move db delete to the postgresql backend lib.
2016-05-22 03:02:39 +01:00
5203fbece2
Move db tagging functions to a new postgresql backend lib.
2016-05-22 02:54:49 +01:00
62bf47d0ed
Warning fix on clang
2016-05-19 20:25:00 +02:00
6e9203faa0
Buildfix on some recent clang versions
2016-05-19 20:17:28 +02:00
7cfc25aeeb
Implement tag deletion.
2016-05-19 20:17:05 +02:00
adcdf56254
Extract some code in a separate function.
2016-05-19 09:48:22 +02:00
43cbad47cb
Fix scan progress messages not clearing up correctly
2016-05-18 22:28:00 +02:00
b52271e2b8
Add an option to limit tagging to a given set only.
2016-05-18 21:18:46 +02:00
9d76957c7f
Allow multiple globs on the command line to tag command.
2016-05-18 10:26:34 +02:00
ad556f4be4
Use new case sensitivity option from glob2regex
...
Don't append crap manually to regular expressions.
2016-05-18 10:25:23 +02:00
6b5fe85571
Make mime_type and mime_charset on the fly
...
For the same motivation explained in the parent commit, make string_refs
on the fly instead of storing them.
2016-05-17 21:12:44 +02:00
8c03ba15bc
Crash fix - make path on the fly
...
Newer versions of gcc have a small string optimizations that makes
moving FileRecordData objects behave incorrectly. When a small string is
moved, string_refs into it become invalid. Making a path on the fly
using the path() method also has the side effect of making
FileRecordData smaller in size.
2016-05-17 20:21:44 +02:00
21c5b3efcb
Allow tagging files by glob
...
Tagging by ID is still possible using the --ids switch, which takes a
comma-separated list of IDs. However only one mode is allowed at time,
so either tag by glob OR tag by ID.
2016-05-17 19:54:05 +02:00
67bdabb548
Allow --tags option in locate command.
...
When a comma separated list of tags is specified, locate searches for
filenames matching the given glob that also belong to all of the
specified tags.
2016-05-16 23:45:08 +02:00
5832ca3374
Move split_tags to common lib.
2016-05-16 09:59:10 +02:00
1192ce7c18
Warning fix on gcc
2016-05-14 16:14:58 +02:00
0f308455ca
Implement tagging files by database id
2016-05-13 20:48:03 +02:00
537023dbfa
Fix build error when passing a vector<string_ref> to query()
2016-05-13 20:46:13 +02:00
94ebfbdfcb
Move git hash into a dedicated .h so less compiling happens after a commit
2016-05-13 10:21:45 +02:00
fde1355653
Enable searching by glob in locate command.
2016-05-12 20:49:11 +02:00
b6953eeaa1
Add unit test for tag splitting
2016-05-05 20:05:56 +02:00
6afc16fe5b
Accept a comma-separated list of tags as the first parameter in tag
2016-05-05 19:10:46 +02:00
da222940ca
Update copyright year in -h message
...
Also make it nicer/easier (hopefully) to remember and to update it next
time.
2016-05-04 19:39:46 +02:00
9ae2474f58
Accept file or set ids on the command line
2016-05-04 10:04:20 +02:00
a508a22af8
New empty action "tag"
2016-05-04 09:43:54 +02:00
265337607e
Fix segfault when no actions are found.
2016-04-30 19:41:00 +02:00
76a289df62
Improve aspect of tab-completion in navigate
2016-04-30 19:40:35 +02:00
76388be56e
Refactoring in listdircontent.
2016-04-29 23:27:56 +02:00
0b6e6ad1d6
Move path composition knowledge into EntryPath.
2016-04-28 22:19:28 +02:00
489d545c60
Rename GenericPath to EntryPath.
2016-04-28 10:20:27 +02:00
b71a94ef70
Implement basic tab completion in navigate.
...
It doesn't work super well and it's missing set number
completion, but it's a step forward.
2016-04-28 00:05:22 +02:00
1ef879e9c1
Define all case combinations of cmake_on/off.
...
This saves having to remember to do STRING(TOUPPER) in every cmake file.
At times that's really not convenient if possible at all.
For example when a top-level project is guessing what option some
subproject is going to use and tries to use it for the builtin options
view. The acutal OPTION statement is not there, and it's not nice to put
a string(toupper) on a variable that seems to come from nowhere (and in
fact it might not be there at all).
2016-04-27 01:29:40 +02:00
804a02c63b
Show state of WITH_NICE_MEDIA_TYPES in dindexer -b
2016-04-27 01:20:12 +02:00
72ce72273d
Put definitions for CMAKE_[ON/OFF] in a common header.
2016-04-27 01:18:38 +02:00
a625f69afd
Import better-enums project from github and put it to use :)
...
Fix build and somewhat improve the dindexer-scan --help message.
2016-04-26 20:47:29 +02:00
c21779adde
Tell needed compiler features to cmake instead of -std=c++11 flag
2016-04-26 20:47:29 +02:00
343c44add5
Add an option to compile the build date into dindexer's binary
2016-04-22 01:35:57 +02:00
abde34e240
Add unit test for cli programs
2016-04-21 00:22:00 +02:00
e3e704c50f
Bugfixes and improvements
2016-04-21 00:20:00 +02:00
653371763b
Treat commands as utf8 when looking for possible mispellings
2016-04-21 00:18:20 +02:00
eaba94fa13
Don't cast malloc.
...
There is no need to cast the result of library functions that return void *;
it makes your code hard to read, adds no value, and can hide a bug if you don't
have a valid prototype in scope.
See http://c-faq.com/malloc/mallocnocast.html
2016-04-19 11:20:04 +02:00
c26cdb97f5
Print compiler id and version in builtin info
2016-04-19 01:04:57 +02:00
e6bfd0ecfd
Merge remote-tracking branch 'gitan/master'
2016-04-19 00:37:57 +02:00
31142a49b3
Implement mispelling suggestions for argument to dindexer
2016-04-19 00:09:38 +02:00
3e130cd346
Import pbl c library
2016-04-18 22:52:35 +02:00
7515705c05
Try to use linereader autocomplete
2016-04-15 15:07:22 +02:00
3a2db550d2
Warning fix on clang
2016-03-18 22:28:58 +01:00
aee6c35ede
Gather more build env info and show them when doing -b
2016-03-17 09:50:52 +01:00
ed30de44cf
Add --version switch to main program.
2016-03-16 18:49:49 +01:00
417459b806
Bugfix and improve behaviour
...
The program now tries to be more strict about the command lines it
accepts. See the comment added into main.c for more details.
Also fixed a bug introduced by a previous commit, and another one
about argv not being null-terminated. For some reasons it never
gave problems until now. But it's fixed now, so yeah.
2016-03-16 18:39:18 +01:00
6991fee7f7
Add support for bash autocomplete in main dindexer
...
I'm not sure this is something that should be in the code, since
compgen -W "delete locate navigate query scan" loc
would do just the same as
dindexer --printactions=loc
so this is a duplicated functionality. However this commit is
self-contained, so it should be easy to revert it, if needed.
2016-03-16 00:01:29 +01:00
c6b0fe49b5
Use getopt_long() to parse command line.
2016-03-15 21:10:52 +01:00
96c6e967a0
Cache result of ls command.
2016-03-15 09:33:12 +01:00
d21a4fa374
Use readline
...
Maybe this should be made optional?
2016-03-15 08:42:01 +01:00
0f7a78b395
Fix paths with spaces not being treated correctly.
2016-03-14 19:39:58 +01:00
8fc45ce694
Buildfix on ARM
2016-03-12 00:47:08 +01:00
859764b9a4
Merge branch 'hashdir_refactoring'
2016-03-12 00:37:28 +01:00
e95cd6cc44
Add a --byhash switch to the locate command to search by content.
...
This is mostly to demonstrate the new task classes.
2016-03-12 00:32:15 +01:00
34ead94c8d
Restore DB writing functionality.
...
This ends the scantask refactoring. It is now possible to scan
directories using the new system. Progress feedback is made
available again, although it could use some improvements.
2016-03-10 18:37:42 +01:00
43c8024b0c
Re-add some progress feedback.
...
It's not multithreaded this time o.O
2016-03-10 09:51:27 +01:00
fe7e9c4af9
Turn around Set's tasks so that they provide a SetRecordDataFull.
2016-03-09 21:26:43 +01:00
fd5be6216a
Namespace shortcut
2016-03-09 19:11:45 +01:00
c4c68e3b5c
Take file size in filesearcher.
...
Like this the filesize task I was about to write becomes
unnecessary.
2016-03-09 18:55:03 +01:00
85cbc76de4
Add GeneralFiller helper task to group tasks together.
2016-03-09 09:25:52 +01:00
5798e0d315
New Mime task.
2016-03-09 09:02:17 +01:00
217eac250b
Include only needed headers
2016-03-08 22:56:23 +01:00
c068f93bac
New ContentType task.
...
Fix the problems I encountered while trying to use the new task.
2016-03-08 19:54:04 +01:00
b357bfb2cc
Use basename() instead of the unclear make_rel_path
2016-03-08 19:09:56 +01:00
250b2df846
Add basename() free function.
2016-03-08 19:09:20 +01:00
63c703f20d
Fix wrong hashes for directories
...
set_listing must sort by abs_path, because the path received by
lower_bound() is an absolute path.
Inside hash_dir() the path passed to append_to_vec() is really
the basename of the current item. This will become more clear
in one of the next commits.
2016-03-08 18:58:47 +01:00
7b0189a119
Make debug output a bit nicer.
2016-03-08 08:51:57 +01:00
d2588d3c7e
Implement hashing task.
...
Also get rid of the ShorFileRecordData and put the new LeanBase
class to use.
2016-03-08 08:48:12 +01:00
d579b311f2
Add ShortFileRecordData struct to hold the file list in DirTree.
2016-03-04 21:57:09 +01:00
1956594c01
Move mediatypes.hpp/cpp to machinery.
...
This was needed so that machinery wouldn't depend on common.
Also moved compatibility.h to helpers include dir.
Ideally mediatypes.hpp will be merged into mediatype.hpp, or there
will be some kind of renaming because the two names are
getting really confusing.
2016-03-04 09:01:11 +01:00
6ddf79fad9
Implement MediaType scan task.
2016-03-03 21:31:33 +01:00
5b3e15c45f
Create DirTree class and start moving code around.
2016-03-03 20:46:30 +01:00
b19720548a
Fix program hanging in release.
...
That great idea of spawning a thread for printing to stdout is
firing back. Temporary fix to get past, but the output looks
really broken now.
2016-02-23 23:28:49 +01:00
b55f42e905
Remove debug output.
2016-02-23 21:05:04 +01:00
be9fc3eb0b
Fix various problems introduced in scan with the last commit.
...
Fixes a crash due to an assertion trying to access a
past-the-end iterator.
Gets content type detection to work, although only after
hashing is done (see comment in main.cpp for details).
Fixes a problem with the array passed to guess_content_type
being not sorted as the function expects.
Adds more assertions.
2016-02-23 18:56:07 +01:00
22614432a9
Detect content type after scanning and save to DB.
2016-02-22 19:44:48 +01:00
2e77e4dc0b
Add Glob class and use that one to help guessing content types.
2016-02-20 01:36:31 +01:00
f31463480f
Add stub code to detect VCDs, plus unit test.
2016-02-19 21:57:43 +01:00
7da13f6677
Add assignment operators.
2016-02-19 21:56:10 +01:00
64b5affa4f
Add guess_content_type() function.
...
WiP.
2016-02-19 10:01:11 +01:00
8a97afd6bf
Use Kakoune::SafePtr to hold the PathName in DirIterator.
2016-02-18 21:00:17 +01:00
bb1148863d
Add typedefs to SetListingView<B>
2016-02-17 08:54:38 +01:00
af80771c2e
Fix the new hash_dir().
...
This fixes a bug that I never noticed in the old code.
This also breaks the progress indicator in Release.
2016-02-16 23:01:42 +00:00
5fbad40fda
Add level offset.
...
The level in the FileRecordData is 0 at the base of the
directory being scanned. However PathName::atom_count() can be
more than that. For example scanning /mnt/cdrom will have
level 0 at the mount point, but atom_count() is 2. So it's
necessary to pass in this offset. This way the code can take
that into account and compute stuff correctly.
2016-02-16 20:18:10 +00:00
5908828390
Implement hash_dir() using DirIterator. WiP
...
Keep the old implementation as well, so they can be easily tested.
Uncomment the USE_LEGACY_HASH_DIR define at the top to build with the
old function.
2016-02-16 20:29:50 +01:00
0a3e469951
Support const/non-const DirIterators and SetListingViews
2016-02-16 18:51:17 +01:00
84a2617c24
Enable interoperability between iterators
2016-02-16 18:12:26 +01:00
b94bf308f5
Prepare to allow for non-const iterators.
...
WiP.
2016-02-12 21:02:26 +01:00
bed191c4fc
Better way to construct SetListingView
2016-02-12 20:14:12 +01:00
2a7b8437ed
Make DirIterator forward only
2016-02-12 19:55:32 +01:00
32d32f63cb
Fix SetListing, but some implementation is still missing
2016-02-12 14:44:32 +01:00
5549e7c97d
Show --builtin even when no actions are found.
2016-02-11 09:18:36 +01:00
c89d4b69d5
Add install target.
2016-02-11 09:07:36 +01:00
6362f0f002
Fix unit test
2016-02-10 09:19:54 +01:00
c4a68356f6
Fix a warning in clang
2016-02-10 09:03:31 +01:00
ada0f1df50
Add SetListingView.
...
I'm having linker errors in gtest, pushing to see
if this also happens on the build server.
2016-02-09 23:23:50 +01:00
346946340d
First implementation of set_listing.
...
Trying to move the depth-first iteration logic into
an iterator-like interface. Work in progress WiP
2016-02-09 21:35:13 +01:00
f20d73bdab
Update year on copyright notice.
2016-02-05 21:02:40 +01:00
58c7e917c5
Buildfix on ARM raspberry pi
2016-02-02 20:56:27 +01:00
c5824c36bd
List directories first in the output of ls.
2016-01-29 23:30:59 +00:00
f40600d6e2
Implement ls into subdirectories
2016-01-29 20:15:16 +01:00
362a5d3128
Modernize AutomemBase and add some assertions
2016-01-28 19:02:15 +01:00
394ae7672a
Optimization - replace inner vector with static mem container in dbsource.
2016-01-28 18:13:13 +01:00
8cbd9560ec
Implement listing sets from the db (ls)
2016-01-27 19:18:10 +01:00
9071b25dec
Rename DirManager to GenericPath.
2016-01-26 23:06:36 +01:00
138f112254
Implement cd and pwd commands
2016-01-26 22:53:30 +01:00
c455582a4e
Provide some wrappers so adding commands is more convenient.
2016-01-22 19:54:32 +00:00
99732f6dd8
Add stub for new navigate command.
2016-01-22 15:55:19 +00:00
588d735593
Add missing $3 for the new app_name parameter.
...
Fixes crash after scanning a disk.
2016-01-22 15:54:39 +00:00
eb618f0842
Remove stray cout.
2016-01-22 09:52:34 +00:00
9ee89b815a
Fix mimetype retrieval and make code a bit more readable.
2016-01-22 09:45:20 +00:00
913cdbd59d
Add app_name column to sets.
2016-01-21 20:30:38 +00:00
9f3b27f0b5
Fix scan hanging after listing directories.
2016-01-21 16:56:54 +00:00
e2438f77fd
Fix inserting new files into db.
2016-01-11 14:20:05 +00:00
4c97105d97
Buildfix - pthread is needed when building in Release.
2016-01-11 12:53:37 +00:00
7e737f56a2
Tell libpqtypes to not make copies of strings it receives.
...
The wrapper now copies boost::string_ref into std::string so
it's always safe to pass a string ref, even when it's not
null-terminated.
2016-01-11 12:53:37 +00:00
fa12bb2d74
Bugfix - don't store absolute paths to files in the DB.
...
This was introduced in a previous commit.
2016-01-11 12:53:37 +00:00
3f48d00ad9
Add new str_path_size() method.
...
Refactor code to allow an easier implementation.
2016-01-11 12:53:37 +00:00
5c0a8e0725
Warning fix in Release.
2016-01-07 15:13:03 +00:00
1077661e50
Implement searching for sets by name.
...
Searching for sets also prints the total count of files and
directories in that set.
2016-01-07 15:13:03 +00:00
2593d46ed7
Now there is a cpp for pq_type_helpers.hpp so move code there.
2016-01-07 14:52:32 +00:00
2655ea5f5c
Add support for std::vectors as arrays.
...
Nested vectors are not supported.
2016-01-07 12:57:06 +00:00
f82659e370
Add move semantics.
2016-01-06 13:12:00 +00:00
6edfb08383
Store mime type and charset in the db.
2016-01-06 02:18:42 +00:00
839b9dd49a
Generate a table scheme with some valid username.
...
Have main prog print the new build time constant.
2016-01-05 14:12:36 +00:00
c7b44f6368
Use parametric queries.
2016-01-05 13:30:17 +00:00
f5356e0c29
Change namespace in machinery to mchlib.
2016-01-05 12:49:27 +00:00
4e29200b47
Move scan's core code into new machinery lib.
2016-01-05 12:35:46 +00:00
487b8efe61
Refactor Indexer so that writing to the db is done outside.
2016-01-05 11:24:23 +00:00
c4a2433a04
Move libpqtypes template helpers out to a new header.
2015-12-31 01:20:56 +00:00
ed0fbac5e1
connection.hpp is a public header.
2015-12-31 01:13:26 +00:00
1532d3c083
Fix reading values from ResultSet/Row.
...
Send correct group_id to the db in a safer way.
2015-12-31 00:38:44 +00:00
9bb5689d48
Remove query_void() methods. query() is just fine.
2015-12-30 10:20:52 +00:00
43bec4711e
Clarify PGresult ownership and lifetime.
2015-12-30 10:15:36 +00:00
c6f034d98e
Replace spirit phoenix with standalone phoenix includes.
2015-12-29 19:02:56 +00:00
a3f44bce8d
Use new parametric query_void() instead of manual insert into sets.
2015-12-29 18:11:15 +00:00
390b69e150
Use parametric sql functions to insert new files.
...
Refactoring so that there are no extra copies of data being inserted.
2015-12-29 17:32:22 +00:00
a91e75829f
Fix libpqtypes as external project.
...
Note: due to a bug in cmake/ninja, this change introduces
problems when building with ninja. See:
https://cmake.org/Bug/view.php?id=15256
2015-12-27 03:09:08 +00:00
d8f3578497
Add libpqtypes from http://libpqtypes.esilo.com/
2015-12-26 22:16:51 +00:00
c409a3e0a2
Make main dindexer show options used to build.
2015-12-26 14:51:49 +00:00
05979e24a2
Add split_mime() function.
2015-12-26 14:50:40 +00:00
148b60ee97
Add retrieving the mime type.
...
This is not being stored to the db yet.
2015-12-15 11:26:04 +00:00
2b7d8a6783
Don't scatter the casted type throughout the code.
2015-12-14 13:05:13 +00:00
45e5b7bc8d
Implement "locate" action.
2015-12-14 13:00:10 +00:00
cc5bb9bc34
Sort commands printed in help output.
...
Also fix const-correctness. Hopefully.
2015-12-14 11:55:22 +00:00
57d925034d
Begin working on the "query" action.
2015-12-13 16:46:42 +00:00
93a5130cc5
Move ValidationError to commonlib.
2015-12-13 16:33:58 +00:00
986f0c8db3
Buildfix and minor changes
2015-12-13 16:14:05 +00:00
bef901c56c
Print git revision in --version.
2015-12-12 20:27:31 +00:00
861ea2c809
Implement deleting sets from the db.
2015-12-12 20:20:57 +00:00
4a0ddb4beb
Hide expand() implementation into settings.cpp.
2015-12-12 20:02:42 +00:00
74afa9f502
Rename settings stuff as it got moved to commonlib.
2015-12-12 17:41:34 +00:00
6e7176be10
Move DBSettings to common lib.
2015-12-12 17:23:50 +00:00
8a4afa241e
Begin work on the new delete action.
2015-12-12 16:53:09 +00:00
10b9be1f85
Print human-readable autodetected media type.
2015-12-10 16:31:11 +00:00
6c30621400
Put some common stuff into a separate library.
2015-12-10 15:33:38 +00:00
a0b87e6a2d
Manage read errors.
...
If instructed to continue on errors, store info correctly in the db.
2015-12-10 12:13:16 +00:00
ed3dea8f2c
More assertions and more explicit code.
2015-12-09 18:36:46 +00:00
3a4c11ad4c
Buildfix when blkid.h is not available.
2015-12-08 16:13:36 +00:00
2cdf0cb3b9
Allow disabling media autodetection at build time.
...
Use DINDEXER_WITH_MEDIA_AUTODETECT.
2015-12-08 16:07:21 +00:00
97de157724
Refactor the new media type autodetection code.
2015-12-08 16:03:30 +00:00
c82b41207e
Enable more warnings (and fix some).
2015-12-08 13:57:12 +00:00
fe2ea40c4f
Implement autodetect set type.
...
Only supports HardDisk, Directory, CdRom, DVD, BluRay for now.
2015-12-08 13:56:46 +00:00
87bc031e65
Rename update action to scan.
2015-12-04 19:40:02 +00:00
20ebc10b0c
Dynamically build the type list for the help message.
2015-12-04 19:32:57 +00:00
8708ef9503
Add dispatcher program.
...
Invoke like dindexer <action> a b c and it will exec
ACTIONS_PATH/dindexer-<action> a b c
or
ACTIONS_PATH/<action>/dindexer-<action> a b c
2015-12-04 16:32:32 +00:00
fca9b94afd
Make one more empty target that only carries include paths.
2015-12-04 16:30:41 +00:00
3899680544
Pull lengthof definition into a different header.
2015-12-04 13:48:20 +00:00
d8be48d2e8
Reorganize source code in directories.
2015-12-04 12:58:13 +00:00
3c1b7108f3
Allow type "C" (for CD-Rom)
2015-12-04 12:30:48 +00:00
5f8756ef1b
Store access time and modify time in the db.
2015-12-01 16:35:46 +00:00
f1c3deea63
Group parameters in a struct when calling filesearcher's callback.
2015-12-01 16:34:22 +00:00
8b9241757d
Don't save to DB if the disk has been added already.
2015-11-27 20:41:36 +00:00
e957fde12c
Enable uppercase hash-to-string conversion.
2015-11-27 20:34:42 +00:00
51d0bfdf7a
Avoid messages clashing on stdout in debug.
...
This quickfix completely disables progress output in debug.
2015-11-27 19:54:03 +00:00
f6c638e6e4
Fix wrong assertion.
2015-11-27 19:49:29 +00:00
56735c9d86
Get the config file path from cmake.
2015-11-13 19:25:32 +00:00
f48626960d
Improve progress output.
2015-11-13 15:45:28 +00:00
f7441292bc
Add optional verbose mode that shows the number of hashed files.
2015-11-12 14:07:26 +00:00
a82ab4a4ed
Bugfix in Release.
2015-11-11 20:13:03 +00:00
49a9702115
Make a macro for verbosity.
2015-11-11 20:12:55 +00:00
911da3bb00
Add column 'type' to sets and take its value on command line.
2015-11-11 20:06:14 +00:00
585c7f45b7
Allow passing set name and search path as command line parameters.
2015-11-11 19:24:10 +00:00
9a25846880
Fix release build.
2015-11-11 17:14:51 +00:00
292e32b38f
Read db connection parameters from a yml file.
2015-11-11 17:08:17 +00:00
413bf0444b
Update DB tables and insert set along with new files.
2015-11-11 13:30:41 +00:00
444d2a2fa3
Limit insertion to 100 records per query, but this is still suboptimal.
2015-11-11 12:35:52 +00:00
4236b2ece8
Fix hashing and db insertion.
2015-11-11 12:28:04 +00:00
bd9ce0ef54
Send data to the database.
2015-11-11 02:01:37 +00:00
a00d30b0ee
Calculate hash for all entries.
2015-11-10 17:48:22 +00:00
1dee8e0f83
Implement SSE2 Tiger hashing.
2015-11-09 16:50:56 +00:00
d07dc9a7da
New Indexer class.
2015-11-06 19:48:38 +00:00
4fcc2cf25c
New make_relative_path() function.
2015-11-06 19:48:18 +00:00
3f8cda06aa
Improve interface and add some getters.
2015-11-06 19:48:02 +00:00
485796db48
Keep paths in a string pool.
2015-11-06 17:57:07 +00:00
109ce9b82a
Use string_ref instead of string and implement join().
2015-11-05 17:22:34 +01:00
