From my experience with Dynamics 365 and a data migration, here's the conception we could consider regarding all data migration. Moreover, this could be an helpful and methodological approach regarding a data migration to the Cloud.
Introduction - What questions to ask before migrating ?
In general, a data migration is a mass movement of data from a source to a destination. The data movement can be performed according to following approaches: the "big bang" approach and the “phase " approach.
The "Big Bang" approach means that the migration takes place only once and we know when it starts and when it ends. This approach requires, in principle, a ”shut down” of the system including the data source to be migrated. On the other hand, the phased approach, although it complicates the migration process because the data migrated to the destination has been modified in the original source, allows smaller pieces of data (and metadata) to be migrated.
Whatever approach is chosen, data migration takes place in a specific scenario, a scenario determined by its technological environment. Before describing our scenario, let us first establish the existing and potential list of options related to data migration.
Data migration options
Without emphasizing the technical aspect of a case, in this case, whether it is a "Cloud" technology (cloud technology) or an "on Premise" technology (on-site technology), there are several execution contexts:
Data migration options in details
A specific scenario is characterized by the following three elements: the data and metadata structure, the technology or platform in which the transfer is constrained and the technological context, and finally, our last element, which is the version and build number of the platform.
Here's one example below - no need to describe all the options (please, see p.6 to 7).
Our selected option
Migration is the migration of data considering the three elements of our definition regarding our option:
The data and metadata structure of the 2011 CRM differs from the data and metadata structure of the D365.
The technology and the platform are different. From a .NET point of view, even if the platforms are based on the .NET Framework, their physical and application architectures differ from each other.
Therefore, their versions and build numbers differ from each other.
Our Generic Framework related to our assumptions
Based on a generic framework, we asked ourselves questions related to the following concepts: From where, how, from what, which tools, what are the phases of data migration, what …
Where does the process of migration come from ? What are the source and the constraints ?
What does the process migrate ?
How to execute the process of migration ?
How do we organize our actions and orchestrate the process ?
Our D365 context: 2 kinds of pressure
Coming from the source of data and the target, the pressure could be double :
Context to the platform "Dynamics CRM" (source) and "Dynamics 365" (target);
Technical constraints regarding the platform "Dynamics 365" (target).
In orange - Data context : data integrity, data volume, data structure, data normalization,...
In green - Platform context : Upgrades and hotfixes of the platform, black box,...
In yellow - Functional context : native and custom functionalities, native versus non-supported functionalities,...
In blue - Code context : dependency nodes between native code (JavaScript, C#) and native SQL request, ...
In grey - CRM Services : D365 Architecture, CRM Asynchronous Services,...
In green - Infrastructures : Physical Resources,...
In purple - Deployment : D365 deployment services,...
In brown - Dev Tools : code source, code optimization,...
Strategies and actions in the 7 milestones
Digging into strategies and actions
Regarding the white paper (see p.11), I intend to focus on the strategy as process and the actions helping the process move forward.
Digging into tools and actions
Defining strategy through actions : from grouping to prioritizing strategy
It is about creating groups that contain each of the subgroups we have given the name “ZONE". Concretely, in Group 1, there could be several "ZONES" or zone 1, zone 2 ... Each of the zones that may include entities.
Throughout these processes of grouping and ordering, a process of entity exclusion was built at the same time. Certain entities should not be considered in the migration process.
Defining strategy through actions : from grouping and prioritizing to excluding strategy
As result, it has become possible to establish two groups of entities: those that are an integral part of the data migration and those that are excluded from the same migration process.
There are many reasons why some entities and therefore some data migration has been set aside. The following is a list of the reasons that led to the exclusion of the said entities or tables:
Entities or tables that are "populated“ by importing our additional configuration (forms, attributes, views, security roles, etc.) and custom code (client code like JavaScript, CRM solution files (”zip")).
Entities or tables that are "populated" by the system in the background (by the synchronous and asynchronous services of the CRM): some are used as logging and others as the source of operation for the CRM services.
Entities or tables that are not ”populated" because they were not in the original system (CRM 2011) and therefore, no need to migrate data in the D365 version.
Entities or tables that are not "populated" because they are new in D365 and did not exist in 2011 version.
The table below presents the list of tables (entities), organized according to the four reasons mentioned above. However, the list is not exhaustive for a clear reason.
Processing data migration through the 7 milestones
Digging into the milestones
There are 7 milestones to consider : creating necessary requirements (number 1), preparing data related to CRM entities and metadata (number 2), transferring CRM security data (number 3), validating CRM security data (number 4), transferring CRM data (number 5), validating CRM data (number 6) and completing CRM data (number 7).
Milestone 1 - Creating the basic requirements : preparing database as a template, preparing a new database as tool containing information related to database "source" and the database "target", extracting SQL/C# native code and CRM entities.
Milestone 2 - Preparing data related to the CRM Organization (DB) : cleaning data (security, system and business data), grouping, prioritizing and excluding entities, handling metadata and data model (deployment of CRM solutions).
Milestone 3 - Transferring CRM Security data to D365 : handling D365 model of security, logging and reporting of new D365 model of security.
Milestone 4 - Validating migrated data security : CRM Deployment and mappings of the users, CRM security data validation, retargeting groups and priorities related to entities, adjusting the code,...
Milestone 5 - Completing CRM data transfer (business and system data) : handling the other data (business and system) after security data, logging and reporting of the D365 model.
Milestone 6 - Validating migrated data : CRM Deployment and mappings of the users, CRM business and system data validation, retargeting groups and priorities related to entities, adjusting the code,...
Milestone 7 - Finalization of data : handling the database, , CRM Deployment, CRM security data validation, CRM business and system data validation,...
Iterating between the milestones
The first iteration occurs between the milestones 3 and 4 : after validating CRM security data (User, Business unit, Team) and data related to CRM security data (milestone 4), a failed validation takes a step backward (milestone 3).
Once the milestone 4 is a success, the second iteration occurs between the milestones 5 and 6 : after validating CRM data (business and system data), a failed validation takes a step backward (milestone 5).
Conclusion
The white paper "7 milestones of a data migration with Dynamics 365" could be a "easy-to-use" roadmap tool.
Regardless of whether the target is on-premise or cloud, I am confident that the 7 milestones should always belong to our journey and could help you to anticipate the milestones.
I hope my white paper also could serve as a communication tool, a high-level document that should help articulate
strategic thinking.
You can download my white paper : "The 7 milestones of a data migration with Dynamics 365".