google-cloud-bigtable

Google Cloud Bigtable Client for Java

Java idiomatic client for Cloud Bigtable.

• Product Documentation
• Client Library Documentation - Data API
• Client Library Documentation - Admin API

Quickstart

If you are using Maven with Bom, add this to your pom.xml file

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>com.google.cloud</groupId>
        <artifactId>libraries-bom</artifactId>
        <version>2.9.0</version>
        <type>pom</type>
        <scope>import</scope>
       </dependency>
     </dependencies>
  </dependencyManagement>

  <dependency>
   <groupId>com.google.cloud</groupId>
   <artifactId>google-cloud-bigtable</artifactId>
  </dependency>

If you are using Maven without Bom, Add this to your dependencies.

  <dependency>
   <groupId>com.google.cloud</groupId>
   <artifactId>google-cloud-bigtable</artifactId>
   <version>1.6.0</version>
  </dependency>

If you are using Gradle, add this to your dependencies

compile 'com.google.cloud:google-cloud-bigtable:1.5.0'

If you are using SBT, add this to your dependencies

libraryDependencies += "com.google.cloud" % "google-cloud-bigtable" % "1.5.0"

Authentication

See the Authentication section in the base directory's README.

Getting Started

Prerequisites

For this tutorial, you will need a Google Cloud Platform Console project with the Cloud Bigtable API enabled. You will need to enable billing to use Google Cloud Bigtable. Follow these instructions to get your project set up. You will also need to set up the local development environment by installing the Google Cloud SDK and running the following commands in command line: gcloud auth login.

About Cloud Bigtable

Cloud Bigtable is Google's NoSQL Big Data database service. It's the same database that powers many core Google services, including Search, Analytics, Maps, and Gmail.

Be sure to activate the Cloud Bigtable API and the Cloud Bigtable Admin API under APIs & Services in the GCP Console to use Cloud Bigtable from your project.

See the Bigtable client library documentation (Admin API and Data API) to learn how to interact with Cloud Bigtable using this Client Library.

Concepts

Cloud Bigtable is composed of instances, clusters, nodes and tables.

Instances

Instances are containers for clusters.

Clusters

Clusters represent the actual Cloud Bigtable service. Each cluster belongs to a single Cloud Bigtable instance, and an instance can have up to 4 clusters. When your application sends requests to a Cloud Bigtable instance, those requests are actually handled by one of the clusters in the instance.

Nodes

Each cluster in a production instance has 3 or more nodes, which are compute resources that Cloud Bigtable uses to manage your data.

Tables

Tables contain the actual data and are replicated across all of the clusters in an instance.

Clients

The Cloud Bigtable API consists of:

Data API

Allows callers to persist and query data in a table. It's exposed by BigtableDataClient.

Admin API

Allows callers to create and manage instances, clusters, tables, and access permissions. This API is exposed by: BigtableInstanceAdminClient for Instance and Cluster level resources.

See BigtableTableAdminClient for table management.

See BigtableDataClient for the data client.

See BigtableInstanceAdminClient for the instance admin client.

See BigtableTableAdminClient for the table admin client.

Calling Cloud Bigtable

The Cloud Bigtable API is split into 3 parts: Data API, Instance Admin API and Table Admin API.

Here is a code snippet showing simple usage of the Data API. Add the following imports at the top of your file:

import com.google.cloud.bigtable.data.v2.BigtableDataClient;
import com.google.cloud.bigtable.data.v2.models.Query;
import com.google.cloud.bigtable.data.v2.models.Row;

Then, to make a query to Bigtable, use the following code:

// Instantiates a client
String projectId = "my-project";
String instanceId = "my-instance";
String tableId = "my-table";

// Create the client.
// Please note that creating the client is a very expensive operation
// and should only be done once and shared in an application.
BigtableDataClient dataClient = BigtableDataClient.create(projectId, instanceId);

try {
  // Query a table
  Query query = Query.create(tableId)
      .range("a", "z")
      .limit(26);

  for (Row row : dataClient.readRows(query)) {
    System.out.println(row.getKey());
  }
} finally {
  dataClient.close();
}

The Admin APIs are similar. Here is a code snippet showing how to create a table. Add the following imports at the top of your file:

import static com.google.cloud.bigtable.admin.v2.models.GCRules.GCRULES;
import com.google.cloud.bigtable.admin.v2.BigtableTableAdminClient;
import com.google.cloud.bigtable.admin.v2.models.CreateTableRequest;
import com.google.cloud.bigtable.admin.v2.models.Table;

Then, to create a table, use the following code:

String projectId = "my-instance";
String instanceId = "my-database";

BigtableTableAdminClient tableAdminClient = BigtableTableAdminClient
  .create(projectId, instanceId);

try {
  tableAdminClient.createTable(
      CreateTableRequest.of("my-table")
        .addFamily("my-family")
  );
} finally {
  tableAdminClient.close();
}

OpenCensus Tracing

Cloud Bigtable client supports OpenCensus Tracing, which gives insight into the client internals and aids in debugging production issues. By default, the functionality is disabled. For example to enable tracing using Google Stackdriver:

If you are using Maven, add this to your pom.xml file

<dependency>
  <groupId>io.opencensus</groupId>
  <artifactId>opencensus-impl</artifactId>
  <version>0.24.0</version>
  <scope>runtime</scope>
</dependency>
<dependency>
  <groupId>io.opencensus</groupId>
  <artifactId>opencensus-exporter-trace-stackdriver</artifactId>
  <version>0.24.0</version>
  <exclusions>
    <exclusion>
      <groupId>io.grpc</groupId>
      <artifactId>*</artifactId>
    </exclusion>
    <exclusion>
      <groupId>com.google.auth</groupId>
      <artifactId>*</artifactId>
    </exclusion>
  </exclusions>
</dependency>

If you are using Gradle, add this to your dependencies

compile 'io.opencensus:opencensus-impl:0.24.0'
compile 'io.opencensus:opencensus-exporter-trace-stackdriver:0.24.0'

If you are using SBT, add this to your dependencies

libraryDependencies += "io.opencensus" % "opencensus-impl" % "0.24.0"
libraryDependencies += "io.opencensus" % "opencensus-exporter-trace-stackdriver" % "0.24.0"

At the start of your application configure the exporter:

import io.opencensus.exporter.trace.stackdriver.StackdriverTraceConfiguration;
import io.opencensus.exporter.trace.stackdriver.StackdriverTraceExporter;

StackdriverTraceExporter.createAndRegister(
  StackdriverTraceConfiguration.builder()
      .setProjectId("YOUR_PROJECT_ID")
      .build());

By default traces are sampled at a rate of about 1/10,000. You can configure a higher rate by updating the active tracing params:

import io.opencensus.trace.Tracing;
import io.opencensus.trace.samplers.Samplers;

Tracing.getTraceConfig().updateActiveTraceParams(
    Tracing.getTraceConfig().getActiveTraceParams().toBuilder()
        .setSampler(Samplers.probabilitySampler(0.01))
        .build()
);

OpenCensus Stats

Cloud Bigtable client supports Opencensus Metrics, which gives insight into the client internals and aids in debugging production issues. Metrics prefixed with cloud.google.com/java/bigtable/ focus on operation level metrics across all of the retry attempts that occurred during that operation. RPC level metrics can be gleaned from gRPC's metrics, which are prefixed with grpc.io/client/.

Available operation level metric views:

• cloud.google.com/java/bigtable/op_latency: A distribution latency of each client method call, across all of it's RPC attempts. Tagged by method name and final response status.

• cloud.google.com/java/bigtable/completed_ops: The total count of method invocations. Tagged by method name. Can be compared to grpc.io/client/completed_rpcs to visualize retry attempts.

• cloud.google.com/java/bigtable/read_rows_first_row_latency: A distribution of the latency of receiving the first row in a ReadRows operation.

• cloud.google.com/java/bigtable/rows_per_op: A distribution of rows read per ReadRows operation across all retry attempts.

• cloud.google.com/java/bigtable/mutations_per_batch: A distribution of mutations per BulkMutation.

By default, the functionality is disabled. For example to enable metrics using Google Stackdriver:

If you are using Maven, add this to your pom.xml file

<dependency>
  <groupId>io.opencensus</groupId>
  <artifactId>opencensus-impl</artifactId>
  <version>0.24.0</version>
  <scope>runtime</scope>
</dependency>
<dependency>
  <groupId>io.opencensus</groupId>
  <artifactId>opencensus-exporter-stats-stackdriver</artifactId>
  <version>0.24.0</version>
  <exclusions>
    <exclusion>
      <groupId>io.grpc</groupId>
      <artifactId>*</artifactId>
    </exclusion>
    <exclusion>
      <groupId>com.google.auth</groupId>
      <artifactId>*</artifactId>
    </exclusion>
  </exclusions>
</dependency>

If you are using Gradle, add this to your dependencies

compile 'io.opencensus:opencensus-impl:0.24.0'
compile 'io.opencensus:opencensus-exporter-stats-stackdriver:0.24.0'

If you are using SBT, add this to your dependencies

libraryDependencies += "io.opencensus" % "opencensus-impl" % "0.24.0"
libraryDependencies += "io.opencensus" % "opencensus-exporter-stats-stackdriver" % "0.24.0"

At the start of your application configure the exporter and enable the Bigtable stats views:

import io.opencensus.exporter.stats.stackdriver.StackdriverStatsConfiguration;
import io.opencensus.exporter.stats.stackdriver.StackdriverStatsExporter;

StackdriverStatsExporter.createAndRegister(
    StackdriverStatsConfiguration.builder()
        .setProjectId("YOUR_PROJECT_ID")
        .build()
);

BigtableDataSettings.enableOpenCensusStats();

Troubleshooting

To get help, follow the instructions in the shared Troubleshooting document.

Transport

Bigtable uses gRPC for the transport layer.

Java Versions

Java 7 or above is required for using this client.

Versioning

This library follows Semantic Versioning.

It is currently in major version zero (0.y.z), which means that anything may change at any time and the public API should not be considered stable.

Contributing

Contributions to this library are always welcome and highly encouraged.

See CONTRIBUTING for more information on how to get started and DEVELOPING for a layout of the codebase.

License

Apache 2.0 - See LICENSE for more information.