IDM-Integrations, the Data Process Manager
- Do you spend all your IT administrators' time on writing scripts to handle your XML data?
- Do you find that you miss out on opportunities to sell your data to customers because you can't easily provide it in the form they need and at the time that they need it?
- Do you find that there are problems in your data because processes have been carried out and you have no record of what was done and when?
IDM-Integrations is the data process toolkit that gives you the solution to all these problems and gives you back the control that you need. Your data and your data-handling scripts will be under control, fast, flexible and therefore more valuable in meeting new business opportunities. It provides data administrators with tools to archive scripts, to manage the editing of new or existing scripts, to launch processes on data and to track the progress of these processes.
Here are some examples of the things that you can do better with Integrations:
- Collect data from different sources and create a relevant dataset,
- Apply a data structure,
- Import data into an editing system,
- Apply QA rules and proof your data,
- Manage your data users and administrate your CMS (configuration files),
- Feed your delivery chains (eg typesetting, data licensing digital/online publishing),
- Comply with licensing datasets and formatting requirements,
- Automate global updates to your content, or generate derived content (such as indexes or derivative products).
We can expand this list very easily, based on the day-to-day experience of data administrators. The range of processes that they perform; the large number of datasets on which they perform them; and the need to meet delivery deadlines all makes the scheduling, management and reuse of existing scripts very difficult. Integrations can cut through these layers of complexity and greatly increase your productivity.
Integrations Components
IDM has introduced Integrations to provide data administrators with tools to archive scripts, to manage the editing of new or existing scripts, to launch an operation and to track progress. Integrations consists of:
- A Software Development Kit (SDK) based on Eclipse and a Tools library of re-usable scripts suitable for writing and testing Integrations processes. It provides a series of scripts capable of transforming data through sequences of discrete operations from very simple ones such as "check all hyphens" to much complex ones such as enhancing the data structure by merging new content into an existing dataset. The SDK comes with technical documentation, help file and debugger,
- An SVN Repository (a standard source code controller) where processes and master files are versioned and stored with authentication and backup capabilities (hosted by IDM),
- A web-based Process Launcher where available processes are listed (permissions apply) and suitable to either run them on demand (launch an operation) and/or schedule them to perform periodic operations (launch an operation every week).
How does it work?
The images on the left show how the three parts of Integrations work together to make data administrators' lives easier.
Processes are a consistent collection of scripts. Processes are written by administrators or external contractors in Perl or Python (Python support to come) within the SDK. The SDK makes script-writing easier and more powerful by assisting the developer with specialized libraries of tasks (such as "send sample by email", "upload/download content from an FTP server","import/export from a given CMS" ).
Scripts are automatically stored and synchronised with the repository, enabling collaborative work: several admin/contractors can work on the same script or set of scripts. Because the repository is an SVN it provides all the standard SVN development security and user-friendly features: simultaneous editing, versioning, back up, tagging, branching.
The process is available in the Integrations launcher, making it visible to any user of Integrations. A user can then run a process - depending on his permissions levels - and follow its progress through the web interface. The user can download the output data. This makes your editorial and Rights sales operations more efficient and more customer-friendly.
Who Should Use Integrations?
Integrations is mainly designed for data administrators to provide them with a consistent, secured and archived environment to run scripts and processes.
However, thanks to its launcher end-user interface, Integrations is used by editors, authors and product managers to run pre-set operations to check, proof or license data. Here are some examples of what users can do with Integrations:
Integrations lets you look at data processing questions in a new way and helps to answer these questions: how can we improve our productivity? how can we improve our creativity? how can we improve our security?
Automating Data Processing Operations
