Database Reference
In-Depth Information
Chapter 5
Importing from Another Data Source
One of the most common tasks you need to perform when working with a new database technology is importing
and syncing data from another data source. This chapter explores what you should consider before beginning that
process. It discusses the processes and tools for importing data into Neo4j and how to select the best process or tool
for a specific situation.
In selecting from among the many ways for importing or synchronizing from another data source, give careful
consideration to your goal. To avoid wasting time on a process or tool that does match the scope of your work, start
from the goal of the data import and work back from that point. Each new version of Neo4j makes the process of
importing data easier, but the tools and processes form only part of the import equation.
Import Considerations
To work backward from your goal, you need to be able to answer the question, “What's the purpose of the data
import?” If your goal is to have only enough representative data to test your application, some of the processes and
tools described in this chapter would be overkill and may be ruled out.
On the other hand, if your application will import existing data from a production data source or if you need
to consider options such as near real-time syncing of data, then you will probably need to consider a mix of tools
to complete the work. Moreover, directly importing data might not always make sense, such that incorporating
data-as-a-service might be a better fit for the goal. Table
5-1
tabulates a number of scenarios that you are apt to
encounter, together with appropriate tools. This chapter presents some guidelines about which tools to use and when.
Table 5-1.
Considerations for Importing Data
Task
Stage
Frequency
Data Size
Tool(s)
Import
Development/Test
One time
>5M
Built-in tools
Migrate
Production
One time
<10M
Built-in tools, programmatic
Import
Production
Scheduled
Varied
Built-in tools, programmatic
Third-party datasource tools
Sync (to Neo4j)
Production
Scheduled
Varied
Built-in tools, programmatic, messaging
queue system
The next section offers some examples of using specific tools for specific jobs in importing or syncing data.
