Backup and Restore

[Robert Stuttaford]

So, backups work great. I can backup my production database no problem,
with no effect on the running system.

But restoring the data on a different transactor (my dev machine) is not so
straight-forward.

So far, this is what I'm doing, but I keep running into an error:

- delete database on local transactor (because otherwise I get
":restore/collision The name '<database>' is already in use by a different
database {:db/error :restore/collision}")
- kill all connected peers (should this be this necessary?)
- restore database
- boot up a peer, which tries to connect on startup (to sync schema)
- get error "Caused by: clojure.lang.ExceptionInfo: :peer/request-failed
:database-deleted {:db/error :peer/request-failed, :request
:start-database}"

At the time of this error, I can see the database listed on
http://datomic-rest:9001/data/local, but attempting to view it causes "Resource
not found."

After many combinations or restarts of the transactor and peers and
database deletes and restores and forcibly killing all my running java
processes, I eventually win through to a working restore.

But I've no idea what causes the success.

Sometimes the output of the restore is:

0 of 1 segments copied.
63 of 63 segments copied.

And sometimes it is:

0 of 0 segments copied.
63 of 63 segments copied.

Not sure why it differs.

Size of database backup on disk is ~700kb.
Production system is using datomic-free-0.8.3524
Development system is using datomic-free-0.8.3561

I haven't yet tried to restore a backup on the production system. Given
this difficulty, I'm concerned about how that's going to go when I finally
have to.

Assistance is most appreciated!

[Robert Stuttaford]

Actually, I've just realised that I don't win through with those steps. The
last time I got things working was because I deleted my development machine
transactor's data directory.

Just did that again, and I'm up with my restored data.

So, to be clear: I can't restore data and then use it without hosing my
data directory.

Now, this is fine for a development machine that has a single database. Not
so good for multiple databases or production systems in general!

This is definitely not working as simply I expected it would :-)

I'd be most happy to answer any questions or perform experiments to get the
Datomic guys the info they need to figure this out.

Post by Robert Stuttaford
So, backups work great. I can backup my production database no problem,
with no effect on the running system.
But restoring the data on a different transactor (my dev machine) is not
so straight-forward.
- delete database on local transactor (because otherwise I get
":restore/collision The name '<database>' is already in use by a different
database {:db/error :restore/collision}")
- kill all connected peers (should this be this necessary?)
- restore database
- boot up a peer, which tries to connect on startup (to sync schema)
- get error "Caused by: clojure.lang.ExceptionInfo: :peer/request-failed
:database-deleted {:db/error :peer/request-failed, :request
:start-database}"
At the time of this error, I can see the database listed on
http://datomic-rest:9001/data/local, but attempting to view it causes "Resource
not found."
After many combinations or restarts of the transactor and peers and
database deletes and restores and forcibly killing all my running java
processes, I eventually win through to a working restore.
But I've no idea what causes the success.
0 of 1 segments copied.
63 of 63 segments copied.
0 of 0 segments copied.
63 of 63 segments copied.
Not sure why it differs.
Size of database backup on disk is ~700kb.
Production system is using datomic-free-0.8.3524
Development system is using datomic-free-0.8.3561
I haven't yet tried to restore a backup on the production system. Given
this difficulty, I'm concerned about how that's going to go when I finally
have to.
Assistance is most appreciated!

[Stuart Halloway]

Hi Robert,

Restoring to the :dev or :free protocols has some issues, caused
partially by the fact that (unlike with the other protocols) storage
is embedded in a transactor.

Fixes and documentation for this specific use case (restoring from
production to dev) are coming in the next release.

Cheers,
Stu

On Wed, Oct 31, 2012 at 11:16 PM, Robert Stuttaford

Post by Robert Stuttaford
Actually, I've just realised that I don't win through with those steps. The
last time I got things working was because I deleted my development machine
transactor's data directory.
Just did that again, and I'm up with my restored data.
So, to be clear: I can't restore data and then use it without hosing my data
directory.
Now, this is fine for a development machine that has a single database. Not
so good for multiple databases or production systems in general!
This is definitely not working as simply I expected it would :-)
I'd be most happy to answer any questions or perform experiments to get the
Datomic guys the info they need to figure this out.

Post by Robert Stuttaford
So, backups work great. I can backup my production database no problem,
with no effect on the running system.
But restoring the data on a different transactor (my dev machine) is not
so straight-forward.
- delete database on local transactor (because otherwise I get
":restore/collision The name '<database>' is already in use by a different
database {:db/error :restore/collision}")
- kill all connected peers (should this be this necessary?)
- restore database
- boot up a peer, which tries to connect on startup (to sync schema)
- get error "Caused by: clojure.lang.ExceptionInfo: :peer/request-failed
:database-deleted {:db/error :peer/request-failed, :request
:start-database}"
At the time of this error, I can see the database listed on
http://datomic-rest:9001/data/local, but attempting to view it causes
"Resource not found."
After many combinations or restarts of the transactor and peers and
database deletes and restores and forcibly killing all my running java
processes, I eventually win through to a working restore.
But I've no idea what causes the success.
0 of 1 segments copied.
63 of 63 segments copied.
0 of 0 segments copied.
63 of 63 segments copied.
Not sure why it differs.
Size of database backup on disk is ~700kb.
Production system is using datomic-free-0.8.3524
Development system is using datomic-free-0.8.3561
I haven't yet tried to restore a backup on the production system. Given
this difficulty, I'm concerned about how that's going to go when I finally
have to.
Assistance is most appreciated!

--
-- You received this message because you are subscribed to the Google Groups
Datomic group. To post to this group, send email to
https://groups.google.com/d/forum/datomic?hl=en

[Robert Stuttaford]

Hi Stuart,

Thanks. I have now upgraded to datomic-free-0.8.3595. Please let me know,
what is the workflow that should work when restoring to datomic:free?

- Do I need to delete the database on the target first? (I know your
documentation says that the target must not have a database already, but
I've been able to run restores with databases in place.)
- Do I need to disconnect all the peers?
- Do I need to restart the transactor?
- I know backup is incremental. Is restore incremental in any way?

I ask because I still have issues restoring a database backed up
on datomic-free-0.8.3524 to datomic-free-0.8.3595. I'm still having to rm
-rf data/*.

Sometimes after killing all processes, restoring, restarting the transactor
(the steam-roller approach), when connecting a peer to the newly restored
database, I get the following error. If I wait a little bit, and retry
again, it goes through:

Exception in thread "main" clojure.lang.ExceptionInfo: Error communicating
with HOST localhost on PORT 4334 {:host "localhost", :alt-host nil, :port
4334, :username "...", :password "..."}
at clojure.core$ex_info.invoke(core.clj:4227)
at datomic.connector$endpoint_error.invoke(connector.clj:41)
at
datomic.connector.TransactorHornetConnector.admin_request(connector.clj:178)
at datomic.peer$create_connection_state$fn__5076.invoke(peer.clj:195)
at datomic.peer$create_connection_state.invoke(peer.clj:187)
at datomic.peer$create_connection$reconnect_fn__5119.invoke(peer.clj:228)
at datomic.peer$create_connection.invoke(peer.clj:229)
at datomic.peer$get_connection$fn__5155.invoke(peer.clj:352)
at datomic.peer$get_connection.invoke(peer.clj:346)
at datomic.peer$connect_uri.invoke(peer.clj:410)
at clojure.lang.Var.invoke(Var.java:415)
at datomic.Peer.connect(Peer.java:63)
at datomic.api$connect.invoke(api.clj:14)
at cognician.server.db.core$conn.invoke(core.clj:15)
at cognician.server.db.core$load_schemas_BANG_.invoke(core.clj:22)
at cognician.server.db.core$update_schema_BANG_.invoke(core.clj:28)
at cognician.server.server$initialize_BANG_.invoke(server.clj:26)
at cognician.server.server$_main.doInvoke(server.clj:32)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.lang.Var.invoke(Var.java:415)
at user$eval1217.invoke(NO_SOURCE_FILE:1)
at clojure.lang.Compiler.eval(Compiler.java:6511)
at clojure.lang.Compiler.eval(Compiler.java:6501)
at clojure.lang.Compiler.eval(Compiler.java:6477)
at clojure.core$eval.invoke(core.clj:2797)
at clojure.main$eval_opt.invoke(main.clj:297)
at clojure.main$initialize.invoke(main.clj:316)
at clojure.main$null_opt.invoke(main.clj:349)
at clojure.main$main.doInvoke(main.clj:427)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at clojure.lang.Var.invoke(Var.java:419)
at clojure.lang.AFn.applyToHelper(AFn.java:163)
at clojure.lang.Var.applyTo(Var.java:532)
at clojure.main.main(main.java:37)
Caused by: clojure.lang.ExceptionInfo: :peer/request-failed Unable to
create log for cognician-5090424e-c420-425f-b961-0e12da13a2a9 {:db/error
:peer/request-failed, :request :start-database}
at clojure.core$ex_info.invoke(core.clj:4227)
at datomic.error$raise.invoke(error.clj:20)
at
datomic.connector.TransactorHornetConnector.admin_request(connector.clj:174)
... 31 more

Thanks
Robert

Post by Stuart Halloway
Hi Robert,
Restoring to the :dev or :free protocols has some issues, caused
partially by the fact that (unlike with the other protocols) storage
is embedded in a transactor.
Fixes and documentation for this specific use case (restoring from
production to dev) are coming in the next release.
Cheers,
Stu
On Wed, Oct 31, 2012 at 11:16 PM, Robert Stuttaford

Post by Robert Stuttaford
Actually, I've just realised that I don't win through with those steps.

The

Post by Robert Stuttaford
last time I got things working was because I deleted my development

machine

Post by Robert Stuttaford
transactor's data directory.
Just did that again, and I'm up with my restored data.
So, to be clear: I can't restore data and then use it without hosing my

data

Post by Robert Stuttaford
directory.
Now, this is fine for a development machine that has a single database.

Not

Post by Robert Stuttaford
so good for multiple databases or production systems in general!
This is definitely not working as simply I expected it would :-)
I'd be most happy to answer any questions or perform experiments to get

the

Post by Robert Stuttaford
Datomic guys the info they need to figure this out.

Post by Robert Stuttaford
So, backups work great. I can backup my production database no problem,
with no effect on the running system.
But restoring the data on a different transactor (my dev machine) is

not

Post by Robert Stuttaford

Post by Robert Stuttaford
so straight-forward.
- delete database on local transactor (because otherwise I get
":restore/collision The name '<database>' is already in use by a

different

Post by Robert Stuttaford

Post by Robert Stuttaford
database {:db/error :restore/collision}")
- kill all connected peers (should this be this necessary?)
- restore database
- boot up a peer, which tries to connect on startup (to sync schema)

:peer/request-failed

Post by Robert Stuttaford

Post by Robert Stuttaford
:database-deleted {:db/error :peer/request-failed, :request
:start-database}"
At the time of this error, I can see the database listed on
http://datomic-rest:9001/data/local, but attempting to view it causes
"Resource not found."
After many combinations or restarts of the transactor and peers and
database deletes and restores and forcibly killing all my running java
processes, I eventually win through to a working restore.
But I've no idea what causes the success.
0 of 1 segments copied.
63 of 63 segments copied.
0 of 0 segments copied.
63 of 63 segments copied.
Not sure why it differs.
Size of database backup on disk is ~700kb.
Production system is using datomic-free-0.8.3524
Development system is using datomic-free-0.8.3561
I haven't yet tried to restore a backup on the production system. Given
this difficulty, I'm concerned about how that's going to go when I

finally

Post by Robert Stuttaford

Post by Robert Stuttaford
have to.
Assistance is most appreciated!

--
-- You received this message because you are subscribed to the Google

Groups

Post by Robert Stuttaford
Datomic group. To post to this group, send email to

send email to
this group at

Post by Robert Stuttaford
https://groups.google.com/d/forum/datomic?hl=en

[Stuart Halloway]

Hi Robert,

I have expanded the docs at http://docs.datomic.com/backup.html, and

Post by Robert Stuttaford
Hi Stuart,
Thanks. I have now upgraded to datomic-free-0.8.3595. Please let me know,
what is the workflow that should work when restoring to datomic:free?
- Do I need to delete the database on the target first? (I know your
documentation says that the target must not have a database already, but
I've been able to run restores with databases in place.)

No, you need do nothing to storage before or after a restore.

Post by Robert Stuttaford
- Do I need to disconnect all the peers?

Strictly speaking no, but there are interesting subtleties to consider
if you leave them running. So for now I am going to say "yes, restart
the peers."

Post by Robert Stuttaford
- Do I need to restart the transactor?

Yes.

Post by Robert Stuttaford
- I know backup is incremental. Is restore incremental in any way?

Restore is not incrementally visible to consumers.

Post by Robert Stuttaford
I ask because I still have issues restoring a database backed up on
datomic-free-0.8.3524 to datomic-free-0.8.3595. I'm still having to rm -rf
data/*.

I am unable to reproduce this, can you post steps?

Post by Robert Stuttaford
Sometimes after killing all processes, restoring, restarting the transactor
(the steam-roller approach), when connecting a peer to the newly restored
database, I get the following error. If I wait a little bit, and retry
Exception in thread "main" clojure.lang.ExceptionInfo: Error communicating
with HOST localhost on PORT 4334 {:host "localhost", :alt-host nil, :port
4334, :username "...", :password "..."}

This is a bug, and for now the workaround is "try again." :-/

Regards,
Stu

[Robert Stuttaford]

Thanks Stuart. I think I wasn't restarting the transactor after restores!

I'm glad the documentation got an improvement :-)

Post by Stuart Halloway
Hi Robert,
I have expanded the docs at http://docs.datomic.com/backup.html, and

Post by Robert Stuttaford
Hi Stuart,
Thanks. I have now upgraded to datomic-free-0.8.3595. Please let me know,
what is the workflow that should work when restoring to datomic:free?
- Do I need to delete the database on the target first? (I know your
documentation says that the target must not have a database already, but
I've been able to run restores with databases in place.)

No, you need do nothing to storage before or after a restore.

Post by Robert Stuttaford
- Do I need to disconnect all the peers?

Strictly speaking no, but there are interesting subtleties to consider
if you leave them running. So for now I am going to say "yes, restart
the peers."

Post by Robert Stuttaford
- Do I need to restart the transactor?

Yes.

Post by Robert Stuttaford
- I know backup is incremental. Is restore incremental in any way?

Restore is not incrementally visible to consumers.

Post by Robert Stuttaford
I ask because I still have issues restoring a database backed up on
datomic-free-0.8.3524 to datomic-free-0.8.3595. I'm still having to rm -rf
data/*.

I am unable to reproduce this, can you post steps?

Post by Robert Stuttaford
Sometimes after killing all processes, restoring, restarting the transactor
(the steam-roller approach), when connecting a peer to the newly restored
database, I get the following error. If I wait a little bit, and retry
Exception in thread "main" clojure.lang.ExceptionInfo: Error communicating
with HOST localhost on PORT 4334 {:host "localhost", :alt-host nil, :port
4334, :username "...", :password "..."}

This is a bug, and for now the workaround is "try again." :-/
Regards,
Stu
--