Mnesia

From Bauman National Library
This page was last modified on 25 June 2016, at 22:09.
Mnesia
Mnesia.png
Repository {{#property:P1324}}
Written in Erlang
Operating system Cross platform
Type RDBMS
License Open Source Erlang License
Website erlang.org/doc/apps/mnesia/

Mnesia is a distributed, soft real-time database management system written in the Erlang. It is distributed as part of the Open Telecom Platform.

General

Mnesia is a layer built on top of ETS and DETS to add a lot of functionality to these two databases. It mostly contains things many developers might end up writing on their own if they were to use them intensively. Features include the ability to write to both ETS and DETS automatically, to both have DETS' persistence and ETS' performance, or having the possibility to replicate the database to many different Erlang nodes automatically.

Another feature we've seen to be useful is transactions. Transactions basically mean that you're going to be able to do multiple operations on one or more tables as if the process doing them were the only one to have access to the tables. This is going to prove vital as soon as we need to have concurrent operations that mix read and writes as part of a single unit. One example would be reading in the database to see if a username is taken, and then creating the user if it's free. Without transactions, looking inside the table for the value and then registering it counts as two distinct operations that can be messing with each other — given the right timing, more than one process at a time might believe it has the right to create the unique user, which will lead to a lot of confusion. Transactions solve this problem by allowing many operations to act as a single unit.

The nice thing about Mnesia is that it's pretty much the only full-featured database you'll have that will natively store and return any Erlang term out of the box (at the time of this writing). The downside of that is that it will inherit all the limitations of DETS tables in some modes, such as not being able to store more than 2GB of data for a single table on disk (this can in fact be bypassed with a feature called fragmentation.)

If we refer to the CAP theorem, Mnesia sits on the CP side, rather than the AP side, meaning that it won't do eventual consistency, will react rather badly to netsplits in some cases, but will give you strong consistency guarantees if you expect the network to be reliable (and you sometimes shouldn't).

Note that Mnesia is not meant to replace your standard SQL database, and it's also not meant to handle terabytes of data across a large number of data centers as often claimed by the giants of the NoSQL world. Mnesia is rather made for smaller amounts of data, on a limited number of nodes. While it is possible to use it on a ton of nodes, most people find that their practical limits seem to center around 10 or so. You will want to use Mnesia when you know it will run on a fixed number of nodes, have an idea of how much data it will require, and know that you will primarily need to access your data from Erlang in ways ETS and DETS would let you do it in usual circumstances.

Mnesia is centered around the idea of using a record to define a table's structure. Each table can thus store a bunch of similar records, and anything that goes in a record can thus be stored in a Mnesia table, including atoms, pids, references, and so on[1].

The following are some of the most important and attractive capabilities provided by Mnesia:

  • A relational/object hybrid data model that is suitable for telecommunications applications.
  • A DBMS query language, Query List Comprehension (QLC) as an add-on library.
  • Persistence. Tables can be coherently kept on disc and in the main memory.
  • Replication. Tables can be replicated at several nodes.
  • Atomic transactions. A series of table manipulation operations can be grouped into a single atomic transaction.
  • Location transparency. Programs can be written without knowledge of the actual data location.
  • Extremely fast real-time data searches.
  • Schema manipulation routines. The DBMS can be reconfigured at runtime without stopping the system.

Key features

Database model

"Rows" in tables are represented as records that contain a key value and a data field. This data field may in turn be a tuple containing an Erlang data structure of any complexity.

Relational features

The database model is relational, but isn't what someone familiar with SQL might expect. A database contains tables. Relationships between them are modelled as other tables.

A key feature of Mnesia's high-availability approach is that tables can be reconfigured within a schema and relocated between nodes, not only while the database is still running, but even while write operations are still going on.

Coding for Mnesia

The query language of Mnesia is Erlang itself, rather than SQL. It permits easy representation of transactions as a natural feature of Erlang by allowing developers to utilize a single language throughout an application.

Transactions

Erlang is a functional language. Mnesia builds on this to obtain ACID transaction support. The functional block which is run as a transaction is a commonplace Erlang construct called a Functional Object (or Fun) and is called by the single Mnesia statement mnesia:transaction(F). This can lead to clearer source code than the paired BEGIN / COMMIT syntax of SQL, and so avoids its problem of unclosed transactions within a procedure.

Again as a result of the functional nature of Erlang, nesting transactions is simple. It's also possible to distribute transactions across multiple nodes (i.e. separate servers). The semantics of using transactions in this way remains consistent, making it easy to write library code that works equally in either context.

General coding style for Mnesia will always use transactions. For performance reasons, it also supports deliberate "dirty operations" which avoid transactions. These compromise the atomicity and the isolation properties of ACID, but offer around 10× more throughput. In addition there are also in-memory alternatives, although these lose the durability property of ACID.

Checkpoints

A checkpoint is a transaction consistent state that spans over one or more tables. When a checkpoint is activated, the system remembers the current content of the set of tables. The checkpoint retains a transaction consistent state of the tables, allowing the tables to be read and updated while the checkpoint is active. A checkpoint is typically used to back up tables to external media, but they are also used internally in Mnesia for other purposes. Each checkpoint is independent and a table can be involved in several checkpoints simultaneously.

Each table retains its old contents in a checkpoint retainer. For performance critical applications, it can be important to realize the processing overhead associated with checkpoints. In a worst case scenario, the checkpoint retainer consumes more memory than the table itself. Also, each update becomes slightly slower on those nodes where checkpoint retainers are attached to the tables.

For each table, it is possible to choose if there is to be one checkpoint retainer attached to all replicas of the table, or if it is enough to have only one checkpoint retainer attached to a single replica. With a single checkpoint retainer per table, the checkpoint consumes less memory, but it is vulnerable to node crashes. With several redundant checkpoint retainers, the checkpoint survives as long as there is at least one active checkpoint retainer attached to each table.

Core Dumps

If Mnesia malfunctions, system information is dumped to file MnesiaCore.Node.When. The type of system information contained in this file can also be generated with the function mnesia_lib:coredump(). If a Mnesia system behaves strangely, it is recommended that a Mnesia core dump file is included in the bug report.

Dumping Tables

Tables of type ram_copies are by definition stored in memory only. However, these tables can be dumped to disc, either at regular intervals or before the system is shut down. The function mnesia:dump_tables(TabList) dumps all replicas of a set of RAM tables to disc. The tables can be accessed while being dumped to disc. To dump the tables to disc, all replicas must have the storage type ram_copies.

The table content is placed in a .DCD file on the disc. When the Mnesia system is started, the RAM table is initially loaded with data from its .DCD file.

Example

This section provides a simplified demonstration of a Mnesia system startup. The dialogue from the Erlang shell is as follows:

unix>  erl -mnesia dir '"/tmp/funky"'
Erlang (BEAM) emulator version 4.9
        
Eshell V4.9  (abort with ^G)
1> 
1> mnesia:create_schema([node()]).
ok
2> mnesia:start().
ok
3> mnesia:create_table(funky, []).
{atomic,ok}
4> mnesia:info().
---> Processes holding locks <--- 
---> Processes waiting for locks <--- 
---> Pending (remote) transactions <--- 
---> Active (local) transactions <---
---> Uncertain transactions <--- 
---> Active tables <--- 
funky          : with 0 records occupying 269 words of mem 
schema         : with 2 records occupying 353 words of mem 
===> System info in version "1.0", debug level = none <===
opt_disc. Directory "/tmp/funky" is used.
use fall-back at restart = false
running db nodes = [nonode@nohost]
stopped db nodes = [] 
remote           = []
ram_copies       = [funky]
disc_copies      = [schema]
disc_only_copies = []
[{nonode@nohost,disc_copies}] = [schema]
[{nonode@nohost,ram_copies}] = [funky]
1 transactions committed, 0 aborted, 0 restarted, 1 logged to disc
0 held locks, 0 in queue; 0 local transactions, 0 remote
0 transactions waits for other nodes: []
ok

In this example, the following actions are performed[2]:

  • Step 1: The Erlang system is started from the UNIX prompt with a flag
    -mnesia dir '"/tmp/funky"'
    , which indicates in which directory to store the data.
  • Step 2: A new empty schema is initialized on the local node by evaluating mnesia:create_schema([node()]). The schema contains information about the database in general. This is explained in detail later.
  • Step 3: The DBMS is started by evaluating mnesia:start().
  • Step 4: A first table is created, called funky, by evaluating the expression mnesia:create_table(funky, []). The table is given default properties.
  • Step 5: mnesia:info() is evaluated to display information on the terminal about the status of the database.

References

Cite error: Invalid <references> tag; parameter "group" is allowed only.

Use <references />, or <references group="..." />
  1. "Mnesia And The Art of Remembering". 
  2. "Mnesia Getting Started".