How consistent can DATOMIC be?

[Matan Safriel]

Hi,

So I gather from this thread, that Datomic is what's called eventually consistent, whereas, if an application wishes to get full consistency (all nodes having the most up-to-date copy of a datum being queried for) then we need to use sync(). Which will only guarantee consistency in that vein, if the underlying storage backend is configured for full consistency. For example, if the underlying storage backend is Cassandra, and it is not configured for full consistency, which is quite common, then Datomic would not behave as consistent even if sync() is used.

Am I very correct in all that?

Thanks,
Matan

[Dustin Getz]

It helps me to think of Datomic as like Git. github.com is the transactor.
Dev laptops are peers. Peers are sometimes a little stale but always
consistent. You can't have two conflicting views of "master" - but you can
have one view thats a little bit ahead of the other. One difference is that
Git peers pull updates but Datomic peers subscribe, so "a little ahead" is
about 10ms if your peers are colocated with the transactor. The sync api
bridges that 10ms gap.

[Daniel Compton]

Hi Matan

The Datomic blog post on sync has a good explanation on some of this:
http://blog.datomic.com/2013/06/sync.html.

The second form of sync takes no arguments, and works via 'ping' of the

transactor. *It promises not to return until all transactions that have
been acknowledged by the transactor at the time sync was called have
arrived at this peer.* Thus if A has successfully committed a transaction
and told B about it, and B then calls sync(), the database returned by sync
will include A's transaction.

I don't know Datomic's internals well enough to say whether the peer needs
to go out to storage to get the latest transactions, but I suspect it
wouldn't in the normal case of a reasonably up-to-date peer. If this is
correct, then in most cases of sync, the transactor will return the
transactions directly out of memory, and the peer won't need to contact the
storage, greatly reducing the likelihood of doing a read for a recently
written value which hasn't been fully replicated.

This docs section on deployment
<http://docs.datomic.com/deployment.html#storage-problems> explains what

If the storage service fails to read or write data, *Datomic will throw

an exception on any data access that needs to read a segment that the
underlying storage cannot return.* [...] For eventually consistent backend
storages (i.e. Cassandra, DynamoDB), the loss may be temporary and
self-resolving (this is frequently the case during node rebalancing).

I can't remember where I heard about this, but with DynamoDB, Datomic uses
the cheaper and faster eventually consistent reads
<http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadConsistency.html>
when looking for a segment (that it knows exists), retrying with a strongly
consistent read if the EC read fails. I don't know much about Cassandra
with Datomic, but it seems plausible that a similar optimisation might
exist there, with reads doing CL=ONE first, then a higher consistency level
(CL=QUORUM ?) afterwards.

For example, if the underlying storage backend is Cassandra, and it is

not configured for full consistency, which is quite common, then Datomic
would not behave as consistent even if sync() is used.

Datomic reads are always consistent with respect to a given basis-t. The
Datomic transactor distributes writes to the peers on the order of 10s of
ms or less. To get a consistent view of every committed transaction, you
can call sync which will block until all transactions acknowledged by the
transactor arrive at the peer. Your queries will now be consistent with all
committed transactions. However, if any segments required by your query are
not in your cache, and aren't able to be returned by your storage, then an
exception will be thrown. *Datomic reads are always consistent if they
succeed, but in the case of storage unavailability and your peer not having
the cached segments then the reads will fail.*

Hope that helps, as I mentioned, my understanding of Datomic internals is
limited so I may have gotten some details wrong, but I think the overall
picture is (hopefully?) right.

--
Daniel

It helps me to think of Datomic as like Git. github.com is the
transactor. Dev laptops are peers. Peers are sometimes a little stale but
always consistent. You can't have two conflicting views of "master" - but
you can have one view thats a little bit ahead of the other. One difference
is that Git peers pull updates but Datomic peers subscribe, so "a little
ahead" is about 10ms if your peers are colocated with the transactor. The
sync api bridges that 10ms gap.
--
You received this message because you are subscribed to the Google Groups "Datomic" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[Matan Safriel]

Hi Daniel,

A big thank you for working to tie it all together, and in particular the
relevant links.
I think I am getting the round picture now.

Matan



On Mon, Jul 17, 2017 at 3:04 AM, Daniel Compton <

Post by Daniel Compton
Hi Matan
http://blog.datomic.com/2013/06/sync.html.

The second form of sync takes no arguments, and works via 'ping' of the

transactor. *It promises not to return until all transactions that have
been acknowledged by the transactor at the time sync was called have
arrived at this peer.* Thus if A has successfully committed a transaction
and told B about it, and B then calls sync(), the database returned by sync
will include A's transaction.
I don't know Datomic's internals well enough to say whether the peer needs
to go out to storage to get the latest transactions, but I suspect it
wouldn't in the normal case of a reasonably up-to-date peer. If this is
correct, then in most cases of sync, the transactor will return the
transactions directly out of memory, and the peer won't need to contact the
storage, greatly reducing the likelihood of doing a read for a recently
written value which hasn't been fully replicated.
This docs section on deployment
<http://docs.datomic.com/deployment.html#storage-problems> explains what

If the storage service fails to read or write data, *Datomic will throw

an exception on any data access that needs to read a segment that the
underlying storage cannot return.* [...] For eventually consistent
backend storages (i.e. Cassandra, DynamoDB), the loss may be temporary and
self-resolving (this is frequently the case during node rebalancing).
I can't remember where I heard about this, but with DynamoDB, Datomic uses
the cheaper and faster eventually consistent reads
<http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadConsistency.html>
when looking for a segment (that it knows exists), retrying with a strongly
consistent read if the EC read fails. I don't know much about Cassandra
with Datomic, but it seems plausible that a similar optimisation might
exist there, with reads doing CL=ONE first, then a higher consistency level
(CL=QUORUM ?) afterwards.

For example, if the underlying storage backend is Cassandra, and it is

not configured for full consistency, which is quite common, then Datomic
would not behave as consistent even if sync() is used.
Datomic reads are always consistent with respect to a given basis-t. The
Datomic transactor distributes writes to the peers on the order of 10s of
ms or less. To get a consistent view of every committed transaction, you
can call sync which will block until all transactions acknowledged by the
transactor arrive at the peer. Your queries will now be consistent with all
committed transactions. However, if any segments required by your query are
not in your cache, and aren't able to be returned by your storage, then an
exception will be thrown. *Datomic reads are always consistent if they
succeed, but in the case of storage unavailability and your peer not having
the cached segments then the reads will fail.*
Hope that helps, as I mentioned, my understanding of Datomic internals is
limited so I may have gotten some details wrong, but I think the overall
picture is (hopefully?) right.
--
Daniel

It helps me to think of Datomic as like Git. github.com is the
transactor. Dev laptops are peers. Peers are sometimes a little stale but
always consistent. You can't have two conflicting views of "master" - but
you can have one view thats a little bit ahead of the other. One difference
is that Git peers pull updates but Datomic peers subscribe, so "a little
ahead" is about 10ms if your peers are colocated with the transactor. The
sync api bridges that 10ms gap.
--
You received this message because you are subscribed to the Google Groups
"Datomic" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "Datomic" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/datomic/R3SrJGfj3LM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
For more options, visit https://groups.google.com/d/optout.