f12d498e40
Move split_tags to core.
2016-07-10 17:12:19 +01:00
4bd7c3515a
Improve lua scripts.
...
Now a "tags" field gets added to the file item. It contains a comma
separated sorted list of the tags to which the item belongs.
2016-07-10 16:52:20 +01:00
9423c4da73
Implement deleting tags by regex.
2016-07-10 14:13:41 +01:00
7451b8dc5d
Implement delete tag by id.
2016-07-10 14:10:39 +01:00
3d7632ef2c
Forgot to wait for batch to complete.
...
And batch was declared in the wrong place anyways -_-
2016-07-10 14:07:15 +01:00
77fb699ec2
Drop unneeded (and wrong) prototype.
2016-07-10 14:02:42 +01:00
de00a4b469
Add and load dele_tag_if_in_set Lua script.
2016-07-10 14:01:43 +01:00
af7443c48b
Implement tagging by regex.
2016-07-10 14:00:51 +01:00
9c2f1d7403
Move tagging by id into a separate file.
2016-07-10 13:59:45 +01:00
7b77913b49
Allow passing a pattern to *scan commands.
2016-07-10 13:57:16 +01:00
01d587b7da
Move redis range to record data struct into a separate file.
2016-07-10 13:56:46 +01:00
d6e95eac8e
Enable tagging by id in Redis backend.
2016-07-09 14:57:58 +01:00
446d173ad3
Fix broken static_assert.
2016-07-09 14:57:30 +01:00
75335cd390
Implement default ctor and move assignment in Script.
2016-07-08 21:36:50 +01:00
678e8c90d1
Revert "Load save and delete scripts upon connection to Redis."
...
This reverts commit 1aacca9e21
2016-07-08 21:35:31 +01:00
004ba83752
Fix find-by-hash function.
2016-07-08 21:32:09 +01:00
5b14823cb8
Implement storing scan result in Redis backend. Yay!
2016-07-08 19:52:55 +01:00
034ebca873
Don't choke on nil replies from Redis.
2016-07-08 19:52:27 +01:00
1aacca9e21
Load save and delete scripts upon connection to Redis.
2016-07-08 16:08:21 +01:00
53f9e72172
Nest postgre settings into node "connection" for consistency with redis.
2016-07-08 15:20:49 +01:00
afae0b6d11
Load Lua script search paths into a SearchPaths.
2016-07-08 15:18:26 +01:00
cc943791ef
core should always be a .so
2016-07-08 15:17:13 +01:00
baddd228fa
find_plugin_by_name() now relies on the new SearchPaths class.
2016-07-08 14:50:51 +01:00
5f5b2b5770
New core library.
2016-07-08 14:49:40 +01:00
e728eb2a9e
Various really small changes.
2016-07-08 14:17:51 +01:00
14587f1cb8
Buildfix when not using cryptopp.
2016-06-30 11:40:05 +01:00
53bb4684b4
Allow disabling use of cryptopp from code.
...
Just comment out the definition of MAKE_SHA1_WITH_CRYPTOPP
and the code will fall back to getting the sha1 from Redis.
2016-06-30 11:37:05 +01:00
8d7d4c5555
Implement Script class.
2016-06-30 11:27:15 +01:00
0efd71a2fa
Refactor connection code.
...
Move all redis connection and libev code out from command.
Command just wraps some useful methods in AsyncConnection now.
2016-06-30 09:35:34 +01:00
cf9ac6b296
Add locking so async communication works correctly.
...
helgrind says it works. I think.
2016-06-29 22:51:45 +01:00
7f25fdb37c
Add support for 'status string' replies from Redis.
2016-06-29 22:49:38 +01:00
40e189cf4d
Add on_connect/on_disconnect hiredis callbacks.
2016-06-21 17:52:13 +01:00
29e2375e3b
Add constructor for opening connection to socket.
2016-06-21 14:20:30 +01:00
c61619eb38
Add code to submit scripts to Redis.
...
This is untested, but I can't test it until I implement other
things first. I also need to do some refactoring, so I need to
commit and make a checkpoint now.
Consider this code as good as if didn't exist.
2016-06-21 14:19:59 +01:00
7957a94980
Move cmake_on_off.h to helpers dir.
2016-06-20 19:04:24 +01:00
92ef212e44
Use async hiredis API.
2016-06-20 18:19:06 +01:00
cdae333e1f
Don't throw if reply is of error type - just put it in Reply.
2016-06-20 18:06:26 +01:00
fabca35b0a
Prefix cmake option DB_OWNER_NAME with DINDEXER_
2016-06-17 09:30:26 +01:00
691bdd1d1e
Use new custom lexical_cast instead of the one from boost.
2016-06-15 22:38:25 +01:00
7ae8b95599
Rename casts.hpp to lexical_cast.hpp
2016-06-15 15:05:25 +01:00
4b30fabbe4
New string_to_tiger function.
...
Comes with unit test that indirectly also tests the
previously added lexical_cast.
2016-06-15 14:57:52 +01:00
874783e8da
Add support for the MATCH and COUNT options for the SCAN commands.
2016-06-14 19:11:13 +01:00
423effa6cb
Set client name in Redis to dindexer_v0.1.5
2016-06-14 19:10:10 +01:00
53835d54b2
Also return content_type.
2016-06-14 11:07:59 +01:00
2a696aa62a
Allow users to selectively build backends.
...
Pass -DDINDEXER_ENABLED_BACKENDS="list,of,backends" to cmake to tell
which backends you want to be built. Currently available backends are
postgresql and redis. Default is postgresql.
Allowed strings in the comma separated list above are the names of the
directories in src/backends.
2016-06-14 11:03:31 +01:00
a5d5ee5af5
Also return content_type.
2016-06-14 10:41:38 +01:00
549ab7b16a
Write implementation for search_file_by_hash().
...
Note that the postgresql backend does something like
"select blah... where hash=$1 limit 1", so it only returns one match.
The SRANDMEMBER used in this implementation also returns one match,
but it's not deterministic I suppose.
2016-06-14 10:39:42 +01:00
9ef9b05014
Move variant types enum out of implem namespace.
...
Also assert that you can get the type you are trying to get before
actually trying.
2016-06-14 09:37:57 +01:00
ab43215f49
Use the right syntax for Redis command.
...
With this commit, scan subcommand can save all it needs into Redis.
2016-06-14 09:12:32 +01:00
ba8e8b9f7b
Add support for char[N] to the run() method.
2016-06-14 09:11:07 +01:00
efd7118189
Add support for single chars to the run() method.
2016-06-14 09:10:45 +01:00
9a69b8de10
Throw if Redis reply contains any error messages.
2016-06-13 17:33:25 +01:00
93e328d77f
Remove unnecessary usage of recursive_wrapper.
2016-06-13 17:27:52 +01:00
316f8f585b
Rename RedisReplyType to just Reply.
2016-06-13 17:00:12 +01:00
533b571771
Implement ZSCAN Redis command.
2016-06-13 16:20:54 +01:00
78eee0e16f
Make the scan command in ScanPairs a template parameter.
2016-06-13 16:20:31 +01:00
9949c273a1
Implement SSCAN Redis command.
2016-06-13 15:38:18 +01:00
731582d8fe
Move V into ValueFetch.
...
No need to explicitly specify the dereferenced type directly to
the scan iterator anymore.
2016-06-13 15:24:26 +01:00
c7eeddabf1
Implement HSCAN Redis command.
2016-06-13 15:14:10 +01:00
e0a82cce2d
Fix SCAN iteration.
2016-06-13 10:57:42 +01:00
c1e79c435b
New ScanIterator class.
...
Not yet tested and only supporting the SCAN command for now, more to
come.
Also includes some refactoring that was needed to make everything work.
2016-06-10 20:33:11 +02:00
04b667485e
Add optional database selection parameter in yml file.
2016-06-10 09:55:37 +02:00
8bdb1f237f
Replace spaces with tabs
2016-06-09 22:55:38 +02:00
49a8e81fed
Restore adding sets into Redis using the new run() method.
...
Partial implementation - not for use in real world cases.
2016-06-09 22:20:22 +02:00
f7a7015c65
Start putting Redis stuff in a wrapper class.
...
Nonworking, but the project builds.
2016-06-09 19:24:08 +02:00
db23dcea3f
Add missing implementation and fix crashes in redis backend.
...
With this you can insert a new set (set only, no files) with dindexer-scan.
2016-06-07 23:13:24 +01:00
2d2c7d0b9e
Merge remote-tracking branch 'origin/dev' into dev
2016-06-07 22:46:09 +01:00
0ab66fa1f1
Stub implementation of a Redis backend.
2016-06-07 22:45:30 +01:00
3fe962a8e5
Make symlink to postgresql backend
...
This is helpful so when running dindexer in a dev environment you don't
need to list all the possible paths to every backend. Symlinks to all
backends will all be in the same place.
2016-06-06 20:24:05 +02:00
624bba121f
Remove hardcoded path to postgresql backend and make the search dynamic
2016-06-06 19:54:20 +02:00
a1490c3096
Make split_tags a bit more generic so I can reuse it
...
I need it in an upcoming commit where I want to split on ':'
2016-06-06 19:52:42 +02:00
d5cf8e21b2
Add cleaning of the new uuid and label properties
2016-06-04 01:13:47 +02:00
2057c797cb
SetRecordData is not really needed anymore so get rid of it
...
Always use SetRecordDataFull now.
2016-06-04 01:13:26 +02:00
7117ff7be2
Save and read label and uuid to the database
2016-06-04 01:03:20 +02:00
55a61526dc
Fix DiscInfo and make it find label and uuid correctly.
...
It's not the mountpoint's inode that I needed, but that of the dev!
2016-06-04 01:02:37 +02:00
f0e67f1c97
Fix wrong block of code
...
Backend plugin was not being initialized due to some copy paste error.
2016-06-03 21:50:45 +02:00
5e1dfe3749
Warning fix in clang.
2016-06-03 20:27:07 +02:00
742d024bfb
Implement label and uuid retrieval for the disk being scanned.
2016-06-03 20:26:54 +02:00
61b0b586cf
Clean up code to handle failed attempts at loading backend plugins.
2016-06-02 21:16:19 +02:00
7de71943ad
Name backend settings section in yml using the backend name itself.
2016-06-02 18:38:40 +02:00
e2275ce5db
Add a backend interface version number for eventual future changes.
...
Note that the interface is still a work in progress and might change
while still keeping the same interface version. This is going to be true
until the first stable release of this project (non beta).
2016-06-02 10:01:19 +02:00
0cb91fb9ca
Put LIMIT number into a common header.
2016-06-02 09:42:32 +02:00
dfe3655296
Re-enable navigate in the postgre plugin.
2016-06-02 09:35:33 +02:00
780cd9b0ca
Re-enable locate in the postgre plugin.
2016-06-01 10:29:23 +02:00
3608d7b76c
Re-enable scan in the postgre plugin.
...
This also adds a public dependency to machinery in the header-only
backend project. This is because the backend does need functionality
from machinery, such as FileRecordData and TigerHash.
2016-05-31 20:03:31 +02:00
8f51f82abc
Re-enable delete subcommand
2016-05-31 10:30:16 +02:00
df2e04a029
Fix tagging.
...
Tag functions are now fully available through the new plugin.
2016-05-27 21:51:15 +02:00
c009c07c9d
Temporarily disable broken stuff
2016-05-27 21:50:09 +02:00
3927f945ca
Move postgresql header files to the src directory
...
They are private, and anyways no client needs to include anything at all
from the various backends.
2016-05-27 21:49:50 +02:00
20b2a8695e
Load postgresql plugin and use that in tag.
...
Only partially implemented.
2016-05-27 20:31:07 +02:00
c97a08860f
Get rid of the DB_BACKEND build setting.
2016-05-27 20:29:32 +02:00
064fc0cf1a
Create intermediate backend lib to help loading backend plugins.
2016-05-27 20:28:26 +02:00
1fd51f75ba
Rename namespace dinbpostgres to dindb.
...
It's so much better!
2016-05-24 09:57:04 +02:00
45fc758915
Print db backend name in --builtin view
2016-05-24 09:57:03 +02:00
8f98495049
Make dindexer-common depend on postgre backend and fix build
2016-05-24 09:57:03 +02:00
fb722719b2
Move postgresql backend's include files to a new db subdir.
...
Fix build and remove unneeded include in tag.cpp
2016-05-24 09:57:03 +02:00
70caa9e26c
Move db dbsource files to the postgresql backend lib and rename namespace.
2016-05-24 09:57:02 +02:00
baa67138eb
Move MaxSizedArray to helpers include dir.
2016-05-24 09:57:02 +02:00
b6d45eee3c
Move db scan functions to the postgresql backend lib.
2016-05-22 03:17:26 +01:00
641261306f
Move db locate functions to the postgresql backend lib.
2016-05-22 03:05:58 +01:00
