All compute resource backup servers and IS controllers within compute zone share the same distributed database (DDB).

It is tmpfs volume mounted at /onappstore/DB folder. The size of the database can be configured in /onappstore/onappstore.conf by changing parameter dbsize=256.

The database is created and populated with data during the IS startup process inside the SAMController service as shown below.

DDB is FS-based data stored in folders as text files.

[FS_DB]  Datastore  Subdir

/onappstore/DB/Datastore: jdhbxnovc7zr86 /onappstore/DB/Datastore/jdhbxnovc7zr86: description   members   name   overcommit   replicas   sector_size   stripes   uuid   vdisks /onappstore/DB/Datastore/jdhbxnovc7zr86/members: 1308341613   1638717502   3789723832   3830220270   4284135282  976785243 /onappstore/DB/Datastore/jdhbxnovc7zr86/vdisks: awoeuctr854pmd   qksozqwm8yaji   l03obar8v249sz   n8k74ulq5my1w2  r6vpuag9l0syn4   t2sbe73qmki1du

[FS_DB] v2 Subdir

This folder contains information about vDisks in JSON format it also stores metadata about these files.
It is a new way of storing data inside DDB, but for now, it is only for snapshot information.
ls /onappstore/DB/v2/VDisk/
12kaws896bte4y.json 12kaws896bte4y.json.mtd 243zhavwnfb6pq.json 243zhavwnfb6pq.json.mtd q9c7ny6jfa842g.json q9c7ny6jfa842g.json.mtd


[FS_DB] locks Subdir


This folder holds locks for resources (nodes, data stores, vDisks, etc.) to prevent transaction interference and race conditions.
In general, operation files are not stored here for a long time. Accordingly, many files inside this folder may indicate problems or overload.
Locks are moved from /tmp folder to DB for better performance and reliability.


[FS_DB] Node Subdir

/onappstore/DB/Node:

1152875907   1308341613   1638717502   1855767384   3789723832 3830220270   4284135282   976785243

/onappstore/DB/Node/1308341613:

backupchannel   datastoremember   freespace   hbtl   hostid   ipaddr  lastping   model   nodestatus   performance   revision   serial   size   status utilization   uuid   vdisks   vendor   version

/onappstore/DB/Node/1308341613/vdisks:

awoeuctr854pmd   n8k74ulq5my1w2   t2sbe73qmki1du


[FS_DB]  VDisk  Subdir

/onappstore/DB/VDisk:

awoeuctr854pmd   qksozqwm8yaji   l03obar8v249sz   n8k74ulq5my1w2   r6vpuag9l0syn4  t2sbe73qmki1du

/onappstore/DB/VDisk/awoeuctr854pmd:

datastore   description   highest_gen_count   memberinfo   membership   name   replicas   sector_size  

sectors   snapshot   snapshots   st_mems   st_size   uuid

/onappstore/DB/VDisk/awoeuctr854pmd/memberinfo:

1308341613   1638717502   4284135282   976785243

/onappstore/DB/VDisk/awoeuctr854pmd/memberinfo/1308341613:

frontend   membership_gen_count   nodestatus   numkeys   port   seqno   snap_limit   st_mem   status

…..

/onappstore/DB/VDisk/awoeuctr854pmd/snapshots:

n8k74ulq5my1w2   t2sbe73qmki1du


DDB stores data about virtual objects used for IS configuration and management. Accordingly, tmpfs are stored in RAM, so we store persistent data at disks attached to IS:

  • Node metadata is stored at the node itself (file: STORAGENODE)

  • Datastore metadata is stored at its members (file: DS-<UUID>)

  • vDisk metadata stored at its members (file: vdisks/<UUID>.xml)

At startup, this persistent data is transfered to /onappstore/DB folder by the local update script. SD daemon syncs it within compute zone via the multicast channel. Syncs are periodical 120-sec timeout or at any metadata changing event like vDisk allocation or online.