ViewInternals

Understanding++

Do you like to know how things work internally? Did you get in trouble as a child for taking apart the vacuum cleaner? No worries, I get it.

 

A deeper understanding

In order to get a deeper understanding of how YapDatabaseView works, and how it integrates with YapDatabase's core architecture, we're going to walk through a few operations, and reveal what the sqlite table actually looks like at each step.

Let's start with an empty database, and then add a few objects to it:

dbConnection.readWrite {(transaction) in

  transaction.setObject(post1, forKey: "bbt", inCollection: "posts")
  transaction.setObject(post2, forKey: "xqy", inCollection: "posts")
  transaction.setObject(post3, forKey: "pxy", inCollection: "posts")

  transaction.setObject(blue, forKey: "bgColor", inCollection: "prefs")
}

Now the database has 3 Post objects, and an item for user preferences. So let's see what it looks like.

(I'm going to attempt to show the SQL table(s), but I'm going to use Dictionary-style markup to show them.)

// The sqlite database (in Dictionary-style markup):
[
  "database": [ // sqlite table
    "posts": [
      "xqy": <post2>,
      "bbt": <post1>,
      "pxy": <post3>
    ],
    "prefs": [
      "bgColor": <blue>
    ]
  ],
  "yap": [ // another sqlite table
    "snapshot": 1
  ]
}

So the sqlite database has 2 tables. One table is named "database", which stores all the objects. And another which is named "yap", which is private, and will be discussed shortly.

By the way, you're encouraged to inspect the sqlite database file that YapDatabase creates. It's a great learning tool. You can use any sqlite file inspector to do so. There are several out there. I like this one.

Next we'll create a YapDatabaseView instance with code that looks something like this:

let grouping = YapDatabaseViewGrouping.withObjectBlock {
  (transaction, collection, key, obj: Any) -> String? in
  
  if let _ = obj as? Post {
    return "posts_all"
  }
  return nil // exclude from view
}

let sorting = YapDatabaseViewSorting.withObjectBlock {
  (transaction, group,
   collection1, key1, obj1: Any,
   collection2, key2, obj2: Any,) -> ComparisonResult in

  let post1 = obj1 as! Post
  let post2 = obj2 as! Post
  
  return post1.comparePostsByTimestamp(post2)
}

let view =
  YapDatabaseView(grouping: grouping
                   sorting: sorting
                versionTag: "theDuckSaysQuack")

database.register(view, withName: "all")

Invoking the synchronous register(_:withName:) function (as we did above) is the equivalent of executing a readWriteTransaction.

So after the view is registered, the database will look something like this:

// The sqlite database (in Dictionary style markup):
[
  "database": [ // sqlite table
    "posts": [
      "xqy": <post2>,
      "bbt": <post1>,
      "pxy": <post3>
    ],
    "prefs": [
      "bgColor": <blue>
    ]
  ],
  "yap": [ // another sqlite table
    "snapshot": 2,
    "view_all": "JSON:{'versionTag':'theDuckSaysQuack'}"
  ]
  "view_all": [ // another sqlite table
    "posts_all" : [
      ("posts","bbt"),
      ("posts","xqy"),
      ("posts","pxy")
    ]
  ]
]

So basically, the YapDatabaseView created another table within the sqlite database. The table name is "view_all", which is a combination of the extension type (YapDatabaseView), and the name the extension was registered with ("all").

When the registerExtension method ran, it enumerated all the posts, and invoked the groupingBlock & sortingBlock in order to calculate the ordered array. Then it stored the array to the database, within its designated sqlite table (view_all).

(Keep in the back of your mind that I'm glossing over a LOT of details here. The on-disk structure of a view is a lot more advanced. It doesn't actually store one big array. It actually splits the array into pages, and stores each individual page. And it actually stores pages of rowid's (Int64), as opposed to collection/key tuples. But we can ignore these details temporarily in order to focus on some primary concepts.)

 

App Relaunch

Now we quit the app and relaunch. And on app launch:

• we initialize our YapDatabase instance
• we initialize and register our YapDatabaseView instance (as before)

So does the YapDatabaseView instance have to do anything?

The answer is NO. When we re-register the YapDatabaseView instance, we tell it that the versionTag is "theDuckSaysQuack". The versionTag is used to signal whether or not the groupingBlock and/or sortingBlock has changed since the last run. And so the view can look in the database and compare the new versionTag with the previous versionTag. And it will see that they are the same, and so the view doesn't is already up-to-date.

Later we execute a readWriteTransaction and add a new post:

dbConnection.readWrite {(transaction) in
  transaction.setObject(post4, forKey: "avz", inCollection: "posts")
}

Recall that a readWrite transaction is atomic. So if you were to add 5 posts in a single transaction, all 5 would be added in one atomic operation. The YapDatabaseView is also a part of this transaction. And when you add a new post, the YapDatabaseView gets to participate in the atomic transaction. And it will automatically update its own tables to match the changes you're making.

So once the readWrite transaction is committed, the database will look something like this:

// The sqlite database (in Dictionary-style markup):
[
  "database": [ // sqlite table
    "posts": [
      "xqy": <post2>,
      "bbt": <post1>,
      "pxy": <post3>,
      "avz": <post4>
    ],
    "prefs": [
      "bgColor": <blue>
    ]
  ],
  "yap": [ // another sqlite table
    "snapshot": 3,
    "view_all": "JSON:{'versionTag':'theDuckSaysQuack'}"
  ],
  "view_all": [ // another sqlite table
    "posts_all": [
      ("posts","bbt"),
      ("posts","xqy"),
      ("posts","pxy"),
      ("posts","avz")
    ]
  ]
]

Notice that post4 was added to the "database" table, and the "view_all" table was automatically updated as well.