Getting Started

This page explains how to get started using Mongo Connector to replicate operations from a MongoDB replica set.

1. Install Mongo Connector. This page explains how.

2. Start up a MongoDB replica set. The MongoDB Manual has a thorough explanation of how to do this. For development purposes, it's probably sufficient to use a single mongod instance:

    $ mongod --replSet singleNodeRepl
    $ mongo
    > rs.initiate()
    # MongoDB is now running on port 27017
   

3. Start up your other database. This example will use Solr:

    java -jar start.jar
    # Solr is now running on port 8983
   

4. Start up Mongo Connector with the DocManager appropriate for your target system:

    mongo-connector -m localhost:27017 -t http://localhost:8983/solr -d solr_doc_manager
   

   old usage (before 2.0 release):

    mongo-connector -m localhost:27017 -t http://localhost:8983/solr -d doc_managers/solr_doc_manager.py
   

At this point, Mongo Connector is tailing your MongoDB replica set's oplog and is ready to replicate to the external database (Solr, in this example).

2014-02-03 22:24:41,016 - INFO - Beginning Mongo Connector
2014-02-03 22:24:42,076 - INFO - Starting new HTTP connection (1): localhost
2014-02-03 22:24:42,082 - INFO - Finished 'http://localhost:8983/solr/admin/luke?show=Schema&wt=json' (get) with body '' in 0.020 seconds.
2014-02-03 22:24:42,088 - INFO - OplogManager: Initializing oplog thread
2014-02-03 22:24:42,089 - INFO - MongoConnector: Starting connection thread MongoClient('localhost', 20000)

When you insert documents into MongoDB, you should see log messages posted by Mongo Connector telling you that everything went ok. Let's insert something now:

mongo
> use foo
> db.c.insert({"name": "Mongo Connector test"})

At this point, we can see a log entry telling us that Mongo Connector successfully replicated the document to Solr:

2014-02-03 22:24:42,107 - INFO - Finished 'http://localhost:8983/solr/update/?commit=true' (post) with body 'u'<add><do' in 0.015 seconds.

We can confirm this by doing a Solr search:

curl -XGET 'http://localhost:8983/solr/select?q=*:*'

And this is what we get back (prettified for easier reading):

<?xml version="1.0" encoding="UTF-8"?>
<response>
  <lst name="responseHeader">
    <int name="status">0</int>
    <int name="QTime">1</int>
    <lst name="params">
      <str name="q">*:*</str>
    </lst>
  </lst>
  <result name="response" numFound="1" start="0">
    <doc>
      <str name="ns">foo.c</str>
      <str name="name">Mongo Connector test</str>
      <long name="_ts">5977053355866849281</long>
      <str name="_id">52f2c25c12c08b48cdb4285a</str>
      <long name="_version_">1459241543025033216</long>
    </doc>
  </result>
</response>

In Solr, you'll notice there are some fields in the resulting document that weren't in the original MongoDB document, including the fields _ts and ns. Other DocManagers keep track of this information, too, but might keep it someplace else. These data track of the timestamp of the oplog entry where this document was last modified and the namespace of the MongoDB collection that holds this document, respectively. These are used internally by Mongo Connector and should not be modified or removed.

Happy connecting!