Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/master' into metrics
Browse files Browse the repository at this point in the history
Conflicts:
	cache.go
	cluster.go
	db.go
	executor.go
	fragment.go
	frame.go
	gossip/gossip.go
	handler.go
	index.go
	server.go
	server/server.go
  • Loading branch information
raskle committed Apr 24, 2017
2 parents 2e3455e + 03aa944 commit 5a8c165
Show file tree
Hide file tree
Showing 55 changed files with 3,540 additions and 3,394 deletions.
4 changes: 2 additions & 2 deletions NOTES
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@

DB Profile
Index Column
┌───────────▼────────────────────────────┐
│0000000000000000000000000000000000000000│
│0000000000000000000000000000000000000000│
│0000000000000000000000000000000000000000│
Bitmap──▶0000000000000000000000000000000000000000│
Row──▶0000000000000000000000000000000000000000│
│0000000000000000000000000000000000000000│
│────────────────────────────────────────┤
│0000000000000000000000000000000000000000│
Expand Down
50 changes: 25 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# pilosa

Pilosa is a bitmap index database.
Pilosa is a bitmap index.

[![Build Status](https://travis-ci.com/pilosa/pilosa.svg?token=Peb4jvQ3kLbjUEhpU5aR&branch=master)](https://travis-ci.com/pilosa/pilosa)

Expand Down Expand Up @@ -123,68 +123,68 @@ Return the version of Pilosa:
$ curl "http://127.0.0.1:10101/version"
```

Return a list of all databases and frames in the index:
Return a list of all indexes and frames in the index:
```sh
$ curl "http://127.0.0.1:10101/schema"
```

### Database and Frame Schema
### Index and Frame Schema

Before running a query, the corresponding database and frame must be created. Note that database and frame names can contain only lower case letters, numbers, dash (`-`), underscore (`_`) and dot (`.`).
Before running a query, the corresponding index and frame must be created. Note that index and frame names can contain only lower case letters, numbers, dash (`-`), underscore (`_`) and dot (`.`).

You can create the database `sample-db` using:
You can create the index `sample-idx` using:

```sh
$ curl -XPOST "http://127.0.0.1:10101/db" \
-d '{"db": "sample-db"}'
$ curl -XPOST "http://127.0.0.1:10101/index" \
-d '{"index": "sample-idx"}'
```

Optionally, you can specify the column label on database creation:
Optionally, you can specify the column label on index creation:

```sh
$ curl -XPOST "http://127.0.0.1:10101/db" \
-d '{"db": "sample-db", "options": {"columnLabel": "user"}}'
$ curl -XPOST "http://127.0.0.1:10101/index" \
-d '{"index": "sample-idx", "options": {"columnLabel": "user"}}'
```

The frame `collaboration` may be created using the following call:

```sh
$ curl -XPOST "http://127.0.0.1:10101/frame" \
-d '{"db": "sample-db", "frame": "collaboration"}'
-d '{"index": "sample-idx", "frame": "collaboration"}'
```

It is possible to specify the frame row label on frame creation:

```sh
$ curl -XPOST "http://127.0.0.1:10101/frame" \
-d '{"db": "sample-db", "frame": "collaboration", "options": {"rowLabel": "project"}}'
-d '{"index": "sample-idx", "frame": "collaboration", "options": {"rowLabel": "project"}}'
```

### Queries

Queries to Pilosa require sending a POST request where the query itself is sent as POST data.
You specify the database on which to perform the query with a URL argument `db=database-name`.
You specify the index on which to perform the query with a URL argument `index=index-name`.

In this section, we assume both the database `sample-db` with column label `user` and the frame `collaboration` with row label `project` was created.
In this section, we assume both the index `sample-idx` with column label `user` and the frame `collaboration` with row label `project` was created.

A query sent to database `sample-db` will have the following format:
A query sent to index `sample-idx` will have the following format:

```sh
$ curl -X POST "http://127.0.0.1:10101/query?db=sample-db" -d 'Query()'
$ curl -X POST "http://127.0.0.1:10101/query?index=sample-idx" -d 'Query()'
```

The `Query()` object referenced above should be made up of one or more of the query types listed below.
So for example, a SetBit() query would look like this:
```sh
$ curl -X POST "http://127.0.0.1:10101/query?db=sample-db" -d 'SetBit(project=10, frame="collaboration", user=1)'
$ curl -X POST "http://127.0.0.1:10101/query?index=sample-idx" -d 'SetBit(project=10, frame="collaboration", user=1)'
```

Query results have the format `{"results":[]}`, where `results` is a list of results for each `Query()`. This
means that you can provide multiple `Query()` objects with each HTTP request and `results` will contain
the results of all of the queries.

```sh
$ curl -X POST "http://127.0.0.1:10101/query?db=sample-db" -d 'Query() Query() Query()'
$ curl -X POST "http://127.0.0.1:10101/query?index=sample-idx" -d 'Query() Query() Query()'
```

---
Expand All @@ -209,17 +209,17 @@ A return value of `{"results":[true]}` indicates that the bit was toggled from 1
A return value of `{"results":[false]}` indicates that the bit was already set to 0 and therefore nothing changed.

---
#### SetBitmapAttrs()
#### SetRowAttrs()
```
SetBitmapAttrs(project=10, frame="collaboration", stars=123, url="http://projects.pilosa.com/10", active=true)
SetRowAttrs(project=10, frame="collaboration", stars=123, url="http://projects.pilosa.com/10", active=true)
```
Returns `{"results":[null]}`

---
#### SetProfileAttrs()
#### SetColumnAttrs()
---
```
SetProfileAttrs(user=10, friends=123, username="mrpi", active=true)
SetColumnAttrs(user=10, friends=123, username="mrpi", active=true)
```

Returns `{"results":[null]}`
Expand All @@ -230,11 +230,11 @@ Returns `{"results":[null]}`
Bitmap(project=10, frame="collaboration")
```
Returns `{"results":[{"attrs":{"stars":123, "url":"http://projects.pilosa.com/10", "active":true},"bits":[1,2]}]}` where `attrs` are the
attributes set using `SetBitmapAttrs()` and `bits` are the bits set using `SetBit()`.
attributes set using `SetRowAttrs()` and `bits` are the bits set using `SetBit()`.

In order to return profile attributes attached to the profiles of a bitmap, add `&profiles=true` to the query string. Sample response:
In order to return column attributes attached to the columns of a bitmap, add `&columnAttrs=true` to the query string. Sample response:
```
{"results":[{"attrs":{},"bits":[10]}],"profiles":[{"user":10,"attrs":{"friends":123, "username":"mrpi", "active":true}}]}
{"results":[{"attrs":{},"bits":[10]}],"columnAttrs":[{"user":10,"attrs":{"friends":123, "username":"mrpi", "active":true}}]}
```

---
Expand Down
2 changes: 1 addition & 1 deletion attr.go
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ func (s *AttrStore) BlockData(i uint64) (map[uint64]map[string]interface{}, erro
return m, nil
}

// txAttrs returns a map of attributes for a bitmap.
// txAttrs returns a map of attributes for an id.
func txAttrs(tx *bolt.Tx, id uint64) (map[string]interface{}, error) {
v := tx.Bucket([]byte("attrs")).Get(u64tob(id))
if v == nil {
Expand Down
6 changes: 3 additions & 3 deletions attr_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (
"github.com/pilosa/pilosa"
)

// Ensure database can set and retrieve profile attributes.
// Ensure database can set and retrieve column attributes.
func TestAttrStore_Attrs(t *testing.T) {
s := MustOpenAttrStore()
defer s.Close()
Expand All @@ -23,14 +23,14 @@ func TestAttrStore_Attrs(t *testing.T) {
t.Fatal(err)
}

// Retrieve attributes for profile #1.
// Retrieve attributes for column #1.
if m, err := s.Attrs(1); err != nil {
t.Fatal(err)
} else if !reflect.DeepEqual(m, map[string]interface{}{"A": int64(100), "B": "VALUE", "C": int64(-27)}) {
t.Fatalf("unexpected attrs(1): %#v", m)
}

// Retrieve attributes for profile #2.
// Retrieve attributes for column #2.
if m, err := s.Attrs(2); err != nil {
t.Fatal(err)
} else if !reflect.DeepEqual(m, map[string]interface{}{"A": int64(200)}) {
Expand Down
25 changes: 15 additions & 10 deletions broadcast.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,11 @@ func (s *StaticNodeSet) Open() error {
return nil
}

func (s *StaticNodeSet) Join(nodes []*Node) error {
s.nodes = nodes
return nil
}

// Broadcaster is an interface for broadcasting messages.
type Broadcaster interface {
SendSync(pb proto.Message) error
Expand Down Expand Up @@ -84,8 +89,8 @@ var NopBroadcastReceiver = &nopBroadcastReceiver{}

const (
MessageTypeCreateSlice = 1
MessageTypeCreateDB = 2
MessageTypeDeleteDB = 3
MessageTypeCreateIndex = 2
MessageTypeDeleteIndex = 3
MessageTypeCreateFrame = 4
MessageTypeDeleteFrame = 5
)
Expand All @@ -95,10 +100,10 @@ func MarshalMessage(m proto.Message) ([]byte, error) {
switch obj := m.(type) {
case *internal.CreateSliceMessage:
typ = MessageTypeCreateSlice
case *internal.CreateDBMessage:
typ = MessageTypeCreateDB
case *internal.DeleteDBMessage:
typ = MessageTypeDeleteDB
case *internal.CreateIndexMessage:
typ = MessageTypeCreateIndex
case *internal.DeleteIndexMessage:
typ = MessageTypeDeleteIndex
case *internal.CreateFrameMessage:
typ = MessageTypeCreateFrame
case *internal.DeleteFrameMessage:
Expand All @@ -120,10 +125,10 @@ func UnmarshalMessage(buf []byte) (proto.Message, error) {
switch typ {
case MessageTypeCreateSlice:
m = &internal.CreateSliceMessage{}
case MessageTypeCreateDB:
m = &internal.CreateDBMessage{}
case MessageTypeDeleteDB:
m = &internal.DeleteDBMessage{}
case MessageTypeCreateIndex:
m = &internal.CreateIndexMessage{}
case MessageTypeDeleteIndex:
m = &internal.DeleteIndexMessage{}
case MessageTypeCreateFrame:
m = &internal.CreateFrameMessage{}
case MessageTypeDeleteFrame:
Expand Down
10 changes: 5 additions & 5 deletions broadcast_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@ import (
func TestMessage_Marshal(t *testing.T) {

testMessageMarshal(t, &internal.CreateSliceMessage{
DB: "d",
Index: "i",
Slice: 8,
})

testMessageMarshal(t, &internal.DeleteDBMessage{
DB: "d",
testMessageMarshal(t, &internal.DeleteIndexMessage{
Index: "i",
})
}

Expand Down Expand Up @@ -47,8 +47,8 @@ func TestBroadcast_BroadcastReceiver(t *testing.T) {
s.BroadcastReceiver = sbr
s.BroadcastReceiver.Start(sbh)

msg := &internal.DeleteDBMessage{
DB: "d",
msg := &internal.DeleteIndexMessage{
Index: "i",
}

s.BroadcastReceiver.(*SimpleBroadcastReceiver).Receive(msg)
Expand Down
Loading

0 comments on commit 5a8c165

Please sign in to comment.