Migrating file shares to AWS using FSx-N: faster & safer

For most large organisations, powerful data management tools are a high priority for better collaboration and strategic planning. So, it’s welcome news that AWS recently announced Amazon FSx for NetApp ONTAP, making migration to advanced automated data solutions faster than ever. 

As an organisation’s data landscape grows, there are crucial decisions to make about how data sharing is managed and supported. Use cases range from full enterprise scale sharing systems to department level exchanges, media & content volumes, archiving and backup.    

Enterprise-scale datasets can be massive and diverse. So, migrating or enabling them from another platform requires careful migration planning and compatibility assessment.   

AWS (Amazon Web Services) offers multiple tools to migrate data to storage targets. This article explains Versent’s best-practice process for migrating on-premises file shares to AWS, using a modern deployment of Amazon FSx for NetApp ONTAP.  

Step 1: assessment & concurrency   

The primary purpose of a modern file server platform is collaboration; optimising data and content sharing. So, the main objective of the initial assessment and discovery phase is to identify the volume of active sessions and concurrency. We need to establish that data, and resource outputs will offer equal or better results when the share is transitioned to the new service. Versent begins this assessment process by creating an inventory of the data at rest and then listing the number of active shares. This will ensure the known concurrency levels can be designed into the future cluster configurations.    

It’s important to understand the baseline of the existing data environment and its potential stress points before measuring it and deciding on specifications for new clusters. This includes: capacity, environmental split, performance demarcation, access multi-pathing and protocols and the profile of data available for user and service consumption.   

Step 2: new cluster deployment    

With assessment completed, we understand how the existing environment operates, and we’re almost ready to deploy the new clusters.    

Growth margins   

It’s important to ensure that storage usage for individual tiers doesn’t exceed the total maximum. FSx-N has a performance tier with a maximum of 192TB (at time of writing). The capacity tier is almost infinite as it’s indefinitely supported by S3.   

The rule of thumb is to cater for 20% active data vs inactive (capacity) data. Therefore, 1PB of capacity volumes would operate with 200TB in performance pools, thereby reaching a logical boundary. (This reasoning doesn’t include any storage optimisation features, such as deduplication, compaction, attrition or compression).   

Placement   

FSx-N environment configuration is flexible, but once the services are deployed into the target subnet, VPC or AWS account, they’re not easily transitioned to a different location. It’s vital to make a decisive choice from the beginning to avoid tedious re-deployment work.   

Deployment principles   

Versent conducts all deployments using Infrastructure as Code best practice principles incorporating zero-touch intervention for resources.    

With the support of CloudFormation, Bash and AWS CLI, cluster deployments can be executed with environment settings according to the requirements dictated by discovery.   

Step 3: integration   

When deployment is completed, integration can begin. Versent ensures that integration has four primary configuration vectors:   

• Networking   
• Authentication   
• Namespaces   
• Shares   

At this stage of the platform setup, we repeat checks on the deployed resources with any available time. If there are anomalies requiring attention, we’ll adjust the environment and recheck the production process.   

Versent configures volumes and shares as a final step in the integration process to match our clients’ requirements. This step is often the most specialised and unique part of cluster provisioning, so we set aside time for due care as it’s a time-consuming procedure.   

Step 4: data migration   

Amazon FSx for NetApp ONTAP has an established pathway for migration, so once a new service is created, data and content can be imported quickly. This is characteristic of AWS managed services, which are usually pre-configured to minimise the time between setup and usage.   

There are three prerequisites for data migration:   

• Migration service credentials must have appropriate access   
• The migration utility – AWS DataSync (SMB only at the time of writing) or NetApp CloudSync w/ Agent via private network access to the source and target environments   
• Bandwidth validation must be performed to ensure that the source services can handle regular replication   

Once the preparatory checks are conducted, replication phases can begin:   

• Replication of initial full data set (without repetition until business unit validation)   
• Scheduled delta data syncs daily or weekly until cutover   
• Monitoring for replication access issues or errors   

To protect services that are already operating, data migration must be managed carefully. Versent begins data migration from existing resources to new cloud environments slowly to ensure that data sovereignty and sensitivity classification is observed.   

Step 5: phased transition vs big bang   

Once a data migration is nearing its synchronisation state, it’s time to transition users and applications into using the new shares. There are two primary options for the format of this transition, each of which has its own merits.   

Phased transition involves the cutover of file system shares over multiple change events. There are three main benefits of this approach:   

• A slower release process means more manageable change control    
• Lower environment testing demands    
• Better protection for performance degradation   

Big bang transition means the cutover of all the shares for a selected filesystem taking place in one change event. The primary benefits of the big bang transition approach are:   

• Efficiency, especially in timeline terms   
• Immediate usage availability   
• Decommissioning value   

Considering the comparative benefits of phased versus big bang transitions, deciding which route to take depends on a few key factors:   

• The impact on the business from the changeover timeline   
• The number of touchpoints, or how many client configurations are connected to an existing service   
• Data volatility; measured hourly or daily   

Step 6: go live   

Getting a new storage environment into service is a substantial forward step for any organisation. The combined utility of AWS and NetApp unifying the work of storage engineers and administrators in an automated, seamless workflow makes this step much more achievable. Using these new tools, going live shrinks from a multi-month build marathon to a process that can be completed in mere days or weeks.   

Versent’s best practice protocols call for multiple dress rehearsals of the go-live action. We do thorough sanity checking of access from all possible access points across a customer’s network and ensure that any test data is correctly populated from the source environment. To ensure this is done thoroughly, we use a storage cutover procedure as a checklist:   

• Enable Amazon FSx for NetApp ONTAP file services to read-write   
• Perform full delta sync before activities begin   
• Communicate with end-users and business stakeholders   
• Stop shares on existing file services   
• Set files to read-only on existing file services   
• Perform secondary full delta sync   
• Enable Amazon FSx for NetApp ONTAP shares   
• Update end-user and application server configurations (DNS or DFS change)   
• Confirm Amazon FSx for NetApp ONTAP services   
• Shutdown old file services   
• Enable targeted backup services on Production Datasets (i.e.: FSx-N Backup)   

Following the above checklist helps to ensure three key outcomes:   

• Data is fully replicated with the latest versions of all items regardless of file server bandwidth   
• Data pollution is avoided   
• Access to FSx-N entry points is carefully controlled, centrally.  

Versent verifies a comprehensive list of procedural checks before embarking on a go-live event. The tactical choice between phased or big bang transition allows engineers and administrators to make safe, secure, gated releases without data degradation. 

Useful takeaways         

By designing, building, deploying, and using Amazon FSx-N for NetApp ONTAP, Versent has accrued significant knowledge about executing successful migrations.     

Building file services by traditional means can be very difficult. Conversely, Amazon FSx for NetApp ONTAP, gives us the ability to deploy more quickly and safely. Considerable care must still be taken in the preparation of the environment, but a shorter deployment process means that we can focus more of our energy on helping application owners and business units adopt their new platform.   

Versent has successfully migrated legacy file services from SMB 1.0 and NFS1. NetApp can support datasets with ACLs and SACLs from either of the above without losing metadata or history.    

Repeated testing helps to avoid environmental performance issues; undersized infrastructure, or inappropriate network placement. This repeated testing is worth the additional effort to avoid later failure or inadequate performance.   

Need a modernised file services solution?      

The introduction of Amazon FSx for NetApp ONTAP makes it possible to run advanced, automated file storage solutions on AWS with virtually no intervention.   

With Versent’s guidance and support, your enterprise can achieve CloudFormation and service provisioning with a single deployment plan.      

Get in touch with a Versent expert to learn more.