Performance tests with big payload (100k):
Redis
30000 requests in 20540 milliseconds, 0.6846666666666666Aerospike
30000 requests in 24860 milliseconds, 0.8286666666666667RocksDB
30000 requests in 52506 milliseconds, 1.7502Performance test payload generator
private String genval(int chars) {
StringBuilder str = new StringBuilder();
for (int i = 0; i != chars; ++ i) {
str.append('a');
}
return str.toString();
}GKVS designed to support startup initialization and remote config. For this purpose it uses scripting configuration based on Lua language.
It gives flexibility to setup endpoints and change confguration on the fly.
Example of Aerospike endpoint:
config = {
hosts = {
{
host="192.168.56.101",
port=3000
}
},
user="",
password=""
}
add_cluster("as1", "aerospike", config );
add_table("test", "as1", { namespace="test", ttl = 100 } );
add_view("TEST", {cluster = "as1", table = "test" });Example of Redis endpoint:
add_cluster("redis1", "redis", { host = "127.0.0.1", port = 6379 } );
add_table("test", "redis1", { ttl = 100 } );
add_view("TEST", { cluster="redis1", table="test" } );Only views are visible for the client. We always can send a script to redirect view to another table.
CRUID performance tests on the same machine:
Redis
30000 requests in 6813 milliseconds, 0.2271Aerospike
30000 requests in 9820 milliseconds, 0.3273333333333333RocksDB
30000 requests in 7446 milliseconds, 0.2482Performance test code
long t0 = System.currentTimeMillis();
for (int i = 0; i != 10000; ++i) {
String key = UUID.randomUUID().toString();
Gkvs.Client.put(TEST, key, new Str("value")).sync();
Gkvs.Client.get(TEST, key).sync().value();
Gkvs.Client.remove(TEST, key);
}
long diff = System.currentTimeMillis() - t0;
System.out.println("30000 requests in " + diff + " milliseconds, " + (double) diff / 30000.0);GKVS support legacy make for development purpose and cmake for deployment purpose in cross-platform systems.
How to build using cmake
mkdir build
cd build
cmake ..
makeIn bare-new system is recommend to use superbuild that pulls all deps from repo
mkdir build
cd build
cmake ../cmake-superbuild/
makeSince last updage of GKVS all connections and channels beween clients ans servers are protected by SSL/TLS. gRPC has the option to leverage SSL in protocol, that option is enforced for GKVS. because Generic Key Value Service is intended to transfer raw data between clients and servers.
GKVS has a special module gkvs-keys. Grab your copy now!
Quick start steps:
Certstrap is not required, but simplifies keys/certs generation. All operations you can do by using OpenSSL. Grab your copy now!
GKVS showed good and acceptable results on peformance tests with enforced SSL. GKVS-java-client has a native implementation of SSL security provider, so makes Java really good on this.
Performance test:
for (int i = 0; i != 10000; ++i) {
String key = UUID.randomUUID().toString();
Gkvs.Client.put(TABLE, key, "value").sync();
Gkvs.Client.get(TABLE, key).sync().value();
Gkvs.Client.remove(TABLE, key);
}Showed good results with enforced SSL:
30000 requests in 10132 milliseconds, 0.33773333333333333That makes GKVS a good candidate for enterprise solutions.
Future is the class in JDK that gives ability to get result somehow in the future. During the interval application can perform another task that how parallelism can give immediate benefit without deep refactoring of application.
Here is the example of GET operation with future:
GkvsFuture<Record> future = Gkvs.Client.get(KEY).async();
// do something else
Record record = future.getUnchecked(); // blocking code And MULTI_GET example:
GkvsFuture<Iterable<Record>> records = Gkvs.Client.multiGet(LOAD_KEYS).async();
// do something else
for (Record rec : records.getUnchecked()) {
System.out.println(rec.key().get());
}GkvsFuture also supports classical get() function with all checked exceptions in the list.
Additionally GkvsFuture has ability to register a listener and get notification when data will be available.
Listener example:
final AtomicBoolean triggered = new AtomicBoolean(false);
GkvsFuture<Record> future = Gkvs.Client.get(KEY).async();
future.addListener(new Runnable() {
@Override
public void run() {
triggered.set(true);
}
});
Record rec = future.getUnchecked();There is an async way to truncate table by scanning all record and sending all keys to remove all operation.
Observer<Key> key = Gkvs.Client.removeAll().async(Observers.<Status>console(done));
Observer<Record> record = Observers.transform(key, Observers.GET_KEY_FN);
Gkvs.Client.scan(TABLE).async(record);How to remove all records in table? Here is the solution:
public void cleanUp(String table) {
Iterator<Record> i = Gkvs.Client.scan(table)
.includeKey(true)
.includeValue(false)
.sync();
while(i.hasNext()) {
try {
Gkvs.Client.remove(i.next().key().get()).sync();
}
catch(GkvsException e) {
e.printStackTrace();
}
}
}GKVS server API uses GRPC (protobuf). This gives ability to write server on C++ and client on Java or any other language.
This is the key-value protocol:
service GenericStore {
rpc get(KeyOperation) returns (ValueResult) {}
rpc multiGet(BatchKeyOperation) returns (BatchValueResult) {}
rpc getAll(stream KeyOperation) returns (stream ValueResult) {}
rpc scan(ScanOperation) returns (stream ValueResult) {}
rpc put(PutOperation) returns (StatusResult) {}
rpc putAll(stream PutOperation) returns (stream StatusResult) {}
rpc remove(KeyOperation) returns (StatusResult) {}
rpc removeAll(stream KeyOperation) returns (stream StatusResult) {}
}Java client works throught GRPC and supports auto-balancing. Performance: 10000 get random requests in 4622 milliseconds. It compiled with all shadow libs to avoid any jar conflicts on applications side.
Java API
Value value = Gkvs.Client.get("TEST", "key").sync().value();
Gkvs.Client.put("TEST", "key", "value").sync();
Gkvs.Client.remove("TEST", "key").sync();
// compare and set
Record record = Gkvs.Client.get("TEST", "key").sync();
boolean updated = Gkvs.Client.put("TEST", "key", "replace_value").compareAndPut(record.version()).sync().updated();
boolean exists = Gkvs.Client.exists("TEST", "key").sync().exists();