- The purpose of the database is to provide a simple NOSQL like storage.
- It should be lightweight and cross platform.
- It should sync over varous channels (containers).
- The containers should be dumb and content agnostic i.e. their content can be encrypted
- Large file content should be handled separately to keep the database lean and data should be loaded lazily
- The sync is simple and inspired by CouchDB, so there will be a deterministic winner in conflict situations
- Works offline
- Blockchain to avoid data corruption
- No single point of failure
- Multiple sync nodes, therefore backup is always in sync
- Simple but robust conflict resolution with no user interaction required
The current working copy of the live database is located somewhere in
~/Application Support where the user has no direct access to it.
A database can store the transactions log and the assets in multiple containers. These can be file packages on the local device or somewhere on the network. But containers can also live in cloud services like Cloud Drive, Dropbox etc.
Both the database and the container have to fit the same
databaseID. Each database instance has its own
instanceID which is used with its entries in the transaction log and assets.
Important actions like data changes are propagated through
In order to get sync working and to have a strategy for conflicts all records build a revision tree. Each
SeaRecord state that is saved gets its own unique
_rev property. Another property called
_revParent is holding the previous
_rev value the change was originating from. This way
_revParent is pointing to the parent node in the revision tree.
Deleted nodes have the property
_deleted = true. (TODO: Or just missing
The strategy for updating is as follows:
- Deleted branches are ignored
- Deeper branches win
- Comparing the
_revproperties the higher value (string compare) wins
A single data item which behaves
NSMutableDictionary like i.e. values can be set like this:
SeaRecord *rec = [[SeaRecord alloc] init]; rec[@"name"] = "John Doe"; rec[@"age"] = @42; [database saveRecord:rec];
But for convenience dynamic properties can be defined as well. Example:
@interface MyRecord : SeaRecord @property NSString *name; @property NSNumber *age; @end @implementation MyRecord @dynamic name; // These are important and required!!! @dynamic age; @end
A record can handle the following object types:
NSData- Small binary data, see
SeaAssetfor large binaries
SeaAsset- Files or other large binary data which should not stay in memory. More
SeaRecordReference- Lightweight reference to another
A local file or data bound to a MIME type is used as the
SeaAsset content. The actual data storage happens when it is used together with a
SeaRecord and then saved to a
SeaDabase. It is very likely that the content is loaded lazily when
.data is accessed.
.URL might also be a remote URL from a
SeaContainer if that is appropriate. Contents and file should never be changed, just create a new
SeaAsset and set it to the
SeaAsset *asset = [[SeaAsset alloc] initURL:url]
Persisted to the database additional meta data will be stored, like
SeaAsset is uniquely identified by the
instance ID and the
index that is also used for storing in the container.
A container can be requested to return all transactions that are newer than the current status. The status is a dictionary of
instanceID keys and the number of the last known
A container can also observe the transactions and emit a change notification. The current database should register for those notifications and trigger a sync.
The containers should always be ready for dumb sync between each other. It has to be "dumb" because the content could be encrypted or shared.
This is the most classic container. It creates the following directory structure:
<ContainerRoot>/ info.json transactions/ <Instance0>/ 1/ // The level of subfolders, each folder has max. 1000 entries 0 // Internally called `index` 1 2 ... <Instance1>/ ... assets/ <Instance0>/ 1/ // The level of subfolders, each folder has max. 1000 entries 0 // Internally called `index` 1 2 ... <Instance1>/ ...
SeaFileSystemContainer but adds file access synchronization to avoid conflicts especially for containers shared via Cloud Drive. It can be used both on macOS and iOS.
Additional safety for the synched data is achieved by block chain inspired writing of data. That means that each new block (see
SeaContainer "transactions" for details) holds the checksum (SHA2 / SHA256) of the previous block. Assets are indirectly checksumed by the meta data stored in the
SeaRecord which again also holds a checksum of the file contents.
SEA/mp 123 1 1 0123.. 0123... ^ ^ ^ ^ ^ ^ ^ 0 1 2 3 4 5 6
- Identification string
- Format specifications like
- Size of data part in bytes
- Index number
- Timestamp, usually a Lamport timestamp
- Previous checksum over complete previous block content including header
- Checksum over data part
If encryption is used it is
AES-256-CBC with a random IV and secured by a HMAC
SHA-512. The password is mangled through PBKDF2 using a random salt and also a HMAC
SHA-512. Over 50,000 iterations are performed.
Any checksum used in the implementation is
SHA-256 which corresponds to a family member of
SHA2, which is pretty well supported cross platform.
A Lamport timestamp is used instead of a regular timestamp to guarantee logical ordering.
This is a utility to store the data locally. For this implementation
SQLite is used, but it is basically a simple Key-Value-Store.
This controller can be used to conveniently feed tables etc. Just set the
database and an optional
recordType and the rest will be behave as expected.