Sea Framework

Sea.objc

SeaDatabase

  • The purpose of the database is to provide a simple NOSQL like storage.
  • It should be lightweight and cross platform.
  • It should sync over varous channels (containers).
  • The containers should be dumb and content agnostic i.e. their content can be encrypted
  • Large file content should be handled separately to keep the database lean and data should be loaded lazily
  • The sync is simple and inspired by CouchDB, so there will be a deterministic winner in conflict situations
  • Works offline
  • Blockchain to avoid data corruption
  • No single point of failure
  • Multiple sync nodes, therefore backup is always in sync
  • Simple but robust conflict resolution with no user interaction required

Implementation

The current working copy of the live database is located somewhere in ~/Application Support where the user has no direct access to it.

A database can store the transactions log and the assets in multiple containers. These can be file packages on the local device or somewhere on the network. But containers can also live in cloud services like Cloud Drive, Dropbox etc.

Both the database and the container have to fit the same databaseID. Each database instance has its own instanceID which is used with its entries in the transaction log and assets.

Notifications

Important actions like data changes are propagated through NSNotificationCenter.

  • SeaRecordDidSaveNotification
  • SeaRecordDidChangeNotification

SeaRevisionTree

In order to get sync working and to have a strategy for conflicts all records build a revision tree. Each SeaRecord state that is saved gets its own unique _rev property. Another property called _revParent is holding the previous _rev value the change was originating from. This way _revParent is pointing to the parent node in the revision tree.

Deleted nodes have the property _deleted = true. (TODO: Or just missing _rev?)

The strategy for updating is as follows:

  1. Deleted branches are ignored
  2. Deeper branches win
  3. Comparing the _rev properties the higher value (string compare) wins

SeaRecord

A single data item which behaves NSMutableDictionary like i.e. values can be set like this:

SeaRecord *rec = [[SeaRecord alloc] init];
rec[@"name"] = "John Doe";
rec[@"age"] = @42;
[database saveRecord:rec];

But for convenience dynamic properties can be defined as well. Example:

@interface MyRecord : SeaRecord
@property NSString *name;
@property NSNumber *age;
@end

@implementation MyRecord
@dynamic name; // These are important and required!!!
@dynamic age;
@end

A record can handle the following object types:

  • NSString
  • NSNumber
  • NSDecimal
  • NSDate
  • NSData - Small binary data, see SeaAsset for large binaries
  • NSDictionary
  • NSArray
  • SeaAsset - Files or other large binary data which should not stay in memory. More
  • SeaRecordReference - Lightweight reference to another SeaRecord entry

SeaAsset

A local file or data bound to a MIME type is used as the SeaAsset content. The actual data storage happens when it is used together with a SeaRecord and then saved to a SeaDabase. It is very likely that the content is loaded lazily when .data is accessed. .URL might also be a remote URL from a SeaContainer if that is appropriate. Contents and file should never be changed, just create a new SeaAsset and set it to the SeaRecord.

SeaAsset *asset = [[SeaAsset alloc] initURL:url]

Persisted to the database additional meta data will be stored, like size, checksum and type. Each SeaAsset is uniquely identified by the instance ID and the index that is also used for storing in the container.

SeaContainer

A container can be requested to return all transactions that are newer than the current status. The status is a dictionary of instanceID keys and the number of the last known index value.

A container can also observe the transactions and emit a change notification. The current database should register for those notifications and trigger a sync.

The containers should always be ready for dumb sync between each other. It has to be "dumb" because the content could be encrypted or shared.

SeaFileSystemContainer

This is the most classic container. It creates the following directory structure:

<ContainerRoot>/
    info.json
    transactions/
        <Instance0>/
            1/           // The level of subfolders, each folder has max. 1000 entries
                0        // Internally called `index`
                1
                2
            ...
        <Instance1>/
        ...
    assets/
        <Instance0>/
            1/           // The level of subfolders, each folder has max. 1000 entries
                0        // Internally called `index`
                1
                2
            ...
        <Instance1>/
            ...

SeaDocumentContainer

Inherits from SeaFileSystemContainer but adds file access synchronization to avoid conflicts especially for containers shared via Cloud Drive. It can be used both on macOS and iOS.

SeaBlockChain

Additional safety for the synched data is achieved by block chain inspired writing of data. That means that each new block (see SeaContainer "transactions" for details) holds the checksum (SHA2 / SHA256) of the previous block. Assets are indirectly checksumed by the meta data stored in the SeaRecord which again also holds a checksum of the file contents.

Internal Format of a Block

SEA/mp 123 1 1 0123.. 0123...
^   ^  ^   ^ ^ ^      ^
0   1  2   3 4 5      6
  1. Identification string SEA
  2. Format specifications like mp for MessagePack
  3. Size of data part in bytes
  4. Index number
  5. Timestamp, usually a Lamport timestamp
  6. Previous checksum over complete previous block content including header
  7. Checksum over data part

Crypto

If encryption is used it is AES-256-CBC with a random IV and secured by a HMAC SHA-512. The password is mangled through PBKDF2 using a random salt and also a HMAC SHA-512. Over 50,000 iterations are performed.

Checksums

Any checksum used in the implementation is SHA-256 which corresponds to a family member of SHA2, which is pretty well supported cross platform.

SeaLamport

A Lamport timestamp is used instead of a regular timestamp to guarantee logical ordering.

SeaStorage

This is a utility to store the data locally. For this implementation SQLite is used, but it is basically a simple Key-Value-Store.

Sea.iOS

xxx

Sea.macOS

SeaNSDocument

xxx

SeaNSArrayController

This controller can be used to conveniently feed tables etc. Just set the database and an optional recordType and the rest will be behave as expected.

Appendix

##

Todo

  • SeaLamport

Related Projects