Migrate to Drupal 8 from a custom site
Ressources
- http://drupal.org/project/drupal
- http://drupal.org/project/migrate_plus
- http://www.drupal.org/project/console
Migrate in Drupal 8
Migrate is now included in the Drupal core for making the upgrade path from 6.x and 7.x versions to Drupal 8.
Drupal 8 has two new modules :
Migrate: « Handles migrations »
Migrate Drupal : « Contains migrations from older Drupal versions. »
None of these module have a User Interface.
« Migrate » contains the core framework classes, the destination, source and process plugins schemas and definitions, and at last the migration config entity schema and definition.
« Migrate Drupal » contains implementations of destination, sources and process plugins for Drupal 6 and 7 you can use it or extend it, it's ready to use. But this module doesn't contain the configuration to migrate all you datas from your older Drupal site to Drupal 8.
The core provides templates of migration configuration entity that are located under each module of the core that needs one, under a folder named 'migration_templates' to find all the templates you can use this command in your Drupal 8 site:
To make a Drupal core to core migration, you will find all the infos here : https://www.Drupal.org/node/2257723 there is an UI in progress for upgrading.
A migration framework
Let have a look at each big piece of the migration framework :
Source plugins
Drupal provides an interface and base classes for the migration source plugin :
- SqlBase : Base class for SQL source, you need to extend this class to use it in your migration.
- SourcePluginBase : Base class for every custom source plugin.
- MenuLink: For D6/D7 menu links.
- EmptySource (id:empty): Plugin source that returns an empty row.
- ...
Process plugins
There is the equivalent of the D7 MigrateFieldHandler but this is not reduced to fields or to a particular field type.
Its purpose is to transform a raw value into something acceptable by your new site schema.
The method transform() of the plugin is in charge of transforming your $value or skipping the entire row if needed.
If the source property has multiple values, the transform() will happen on each one.
Drupal provides migration process plugin into each module of the core that needs it (for the core upgrade),
To find out which one and where it is located you can use this command :
Destination plugins
Destination plugins are the classes that handle where your data are saved in the new Drupal 8 sites schemas.
Drupal provides a lot of useful destination classes :
- DestinationBase : Base class for migrate destination classes.
- Entity (id: entity) : Base class for entity destinations.
- Config (id: config) : Class for importing configuration entities.
- EntityBaseFieldOverride (id: entity:base_field_override): Class for importing base field.
- EntityConfigBase : Base class for importing configuration entities.
- EntityImageStyle (id: entity:image_style): Class for importing image_style.
- EntityContentBase (id: entity:%entity_type): The destination class for all content entities lacking a specific class.
- EntityNodeType: (id: entity:node_type): A class for migrate node type.
- EntityFile (id: entity:file): Class for migrate files.
- EntityFieldInstance: Class for migrate field instance.
- EntityFieldStorageConfig: Class for migrate field storage.
- EntityRevision, EntityViewMode, EntityUser, Book...
- And so more…
Builder plugins:
"Builder plugins implement custom logic to generate migration entities from migration templates. For example, a migration may need to be customized based on the data that is present in the source database; such customization is implemented by builders." - doc API
This is used in the user module, the builder create a migration configuration entity based on a migration template and then add fields mapping to the process, based on the data in the source database. (@see /Drupal/user/Plugin/migrate/builder/d7/User)
Id map plugins:
"It creates one map and one message table per migration entity to store the relevant information." - doc API
This is where rollback, update and the map creation are handled.
Drupal provides the Sql plugin (@see /Drupal/migrate/Plugin/migrate/id_map/Sql) based on the core base class PluginBase.
And we are talking only about core from the beginning.
All the examples (That means docs for devs) are in core !
About now :
While there *almost* a simple UI to use migration in Drupal 8 for Drupal to Drupal, Migrate can be used for every kind of data input. The work is in progess for http://Drupal.org/project/migrate_plus to bring an UI and more source plugins, process plugins and examples. There already is the CSV source plugin and a pending patch for the code example. The primary goal of « migrate plus » is to have all the features (UI, Sources, Destinations.. ) of the Drupal 7 version.
Concrete migration
(migration with Drupal 8 are made easy)
I need to migrate some content with image, attached files and categories from custom tables in an external SQL database to Drupal.
To begin shortly :
- Drush 8 (dev master) and console installed.
- Create the custom module (in the code, I assume the module name is “example_migrate”):
$ Drupal generate:module
or create the module by yourself, you only need the info.yml file. - Activate migrate and migrate_plus tools
$ Drupal module:install migrate_tools
or
$ drush en migrate_tools - What we have in Drupal for the code example :
- a taxonomy vocabulary : ‘example_content_category’
- a content type ‘article’
- some fields: body, field_image, field_attached_files, field_category
- Define in settings.php, the connexion to your external database:
We are going to tell migrate source to use this database target. It happens in each migration configuration file, it’s a configuration property used by the SqlBase source plugin:
This is one of the reasons SqlBase has a wrapper for select query and you need to call it in your source plugin, like $this->select(), instead of building the query with bare hands.
N.B. Each time you add a custom yml file in your custom module you need to uninstall/reinstall the module for the config/install files to imports. In order to avoid that, you can import a single migration config file by copy/paste in the admin/config configuration synchronisation section.
The File migration
The content has images and files to migrate, I suppose in this example that the source database has a unique id for each file in a specific table that hold the file path to migrate.
We need a migration for the file to a Drupal 8 file entity, we write the source plugin for the file migration:
File: src/Plugin/migrate/source/ExampleFile.php
We have the source class and our source fields and each row generate a path to the file on my local disk.
But we need to transform our external file path to a local Drupal public file system URI, for that we need a process plugin. In our case the process plugin will take the external filepath and filename as arguments and return the new Drupal URI.
File: src/Plugin/migrate/process/ExampleFileUri.php
We need another process plugin to transform our source date values to timestamp (created, changed), as the date format is the same across the source database, this plugin will be reused in the content migration for the same purpose:
File: src/Plugin/migrate/process/ExampleDate.php
For the destination we use the core plugin: entity:file.
Now we have to define our migration config entity file, this is where the source, destination and process (field mappings) are defined:
File: config/install/migrate.migration.example_file.yml
We are done for the file migration, you can execute it with the migrate_tools (of the migrate_plus project) drush command:
The Term migration
The content has categories to migrate.
We need to import them as taxonomy term, in this example I suppose the categories didn't have unique ids, it is just a column of the article table with the category name…
First we create the source :
File: src/Plugin/migrate/source/ExampleCategory.php
And we can now create the migration config entity file :
File: config/install/migrate.migration.example_category.yml
This is done, to execute it :
The Content migration
The content from the source has an html content, raw excerpt, image, attached files, categories and the creation/updated date in the format Y-m-d H:i:s
We create the source plugin:
File: src/Plugin/migrate/source/ExampleContent.php
Now we can create the content migration config entity file :
File: config/install/migrate.migration.example_content.yml
Finally, execute it :
Group the migration
Thanks to migrate_plus, you can specify a migration group for your migration.
You need a to create a config entity for that :
File: config/install/migrate_plus.migration_group.example.yml
Then in your migration config yaml file, be sure to have the line migration_group next to the label:
So you can use the command to run the migration together, and the order of execution will depend on the migration dependencies:
I hope that you enjoyed our article.
Best regards,
