Import media files

Media can be imported to the VIP File System with VIP-CLI. This is especially useful for migrating from an existing WordPress site to a site hosted on the VIP Platform.

Limitations

• Media can only be imported to a production environment. Importing media to non-production environments is not necessary, as media uploaded to production is immediately made available to the child environments as well.
• Only one import per environment can be run at a single time. Attempts to start a second import while a previous import is still in progress will result in an error.
• Files imported with the VIP-CLI command will not appear in a site’s WordPress media library unless the attachment data for the files is also imported to the database. The vip import media VIP-CLI command only imports files to the VIP File System.
• After a file has been imported, it cannot be deleted from the VIP File System.

Directory structure

The directory to be imported must be named uploads.

In most cases, media uploaded to a WordPress install is stored in a directory named uploads/, which is a subdirectory of wp-content/. To prepare media files for import to VIP, an archived copy of the uploads/ directory is recommended.

If a non-VIP WordPress site has customized the name of the uploads/ directory, extra steps will be needed to duplicate the directory to a working location and re-name the duplicate directory to uploads before importing the media to VIP.

When an archive’s directory structure contains a directory named uploads/, only the files within the uploads/ directory will be imported. All other files will be ignored.

If no directories named uploads/ exist within an archive, the import tool will not import the contents of the archive into an environment’s uploads/ directory, and will instead import the contents with their existing directory structure.

Multisites

Media import directories for multisites must also be named uploads.

The sub-directory structure of the uploads/ directory for sites belonging to a WordPress multisite will differ from WordPress single sites. The uploads/ file path for the main site (ID 1) of the multisite will be uploads/, but all subsequent sites on a multisite network will be in the uploads/sites/ directory structure, in their own subfolder based on the site number, designated by the database.

Upload paths for all sites other than the main site (ID 1) follow the pattern uploads/sites/{SITE ID}/{YYYY}/{MM}/{FILE NAME}. The values for {SITE ID} do not have any leading 0’s (i.e., use 2, not 02).

Special characters in filenames

WordPress Core sanitizes file names, removing or replacing special characters upon upload to the media library, for example:
 ~?,[]/\=<>:;'\&$#*()|~`!{}%+ and whitespace ().

Media file imports to VIP bypass the WordPress sanitize_file_name() function, and all special characters are allowed, for example:
spaces ( ) [ ] ~ + & # % = ' ” \ < > : ; , / ? $ * | ` ! { } .
Encoded or alternate whitespace, such as %20or + will be converted to proper spaces during the import.

Intermediate images

Traditional installations of WordPress create intermediate images; an array of smaller versions of each uploaded image. The generated intermediate files are unnecessary on VIP, as the VIP File System is able to generate and cache any requested dimension of the originally uploaded image. This reduces the amount of storage a site requires and provides image size flexibility at no loss of speed. It is recommended that intermediate images are excluded from the imported media directory.

If needed, intermediate images can be imported using the -i flag with the import media command. Including intermediate images in an import will cause the process to take a longer time to complete.

Unsupported file types

Imports to the VIP File System will fail for the following file types:

• Files without extensions
• Files not included in the list of file types supported by the VIP File System
• Hidden files or files within hidden directories will be ignored. Hidden files will not be included in the count of files that are discovered by the import media command, and they will not be included in the final count of files that are successfully imported.
• File paths that are more than 255 characters in length. This limitation includes the /wp-content/uploads/ prefix in every imported file path.
• File paths that end with a forward-slash (i.e., /).

SVGs

SVG files allow embedded JavaScript, which can pose a security risk. If SVGs exist in the uploads/ directory that will be imported, it is recommended to scan the file contents prior to media import. The file contents of an SVG can be scanned by:

• examining the contents within a file editor (Visual Studio Code, for example)
• a tool such as http://svg.enshrined.co.uk/ or https://github.com/Automattic/vip-go-svg-sanitizer
• a PHPCS scan using the WordPress-VIP-Go standard

All SVG files introduced to a GitHub repository in a pull request will undergo an SVG analysis as part of the automated PHPCS scan using the WordPress-VIP-Go standard from the VIP Code Analysis Bot. Any discovered irregularities will be reported in the automated code review.

Import media files with VIP-CLI

Prior to running any import commands, confirm all items in this checklist:

• Files for import must be archived (.tar.gz, .tgz, or .zip).
• The size of the entire archive that holds the files to be imported cannot be more than 60 GB.
• The maximum size for an individual file imported to VIP’s File System is 1GB.
• The archived file must decompress to a directory named uploads , and only contain the media required for a site(s).
• Upload archived files to a publicly accessible URL (e.g., https://www.example.com/uploads.tar.gz).
  • File-sharing services such as Dropbox, Box, or WeTransfer are able to provide publicly accessible URLs.
  • A downloadable Dropbox URL must include a ?dl=1 query parameter.
  • Google Drive URLs are not supported.

If the file for import does not meet the above requirements, create a VIP Support ticket requesting assistance with the import and include a downloadable link for the archived file.

Import media

Import media to the VIP File System with the vip import media VIP-CLI command.

Refer to the help menu for a full list of commands, options, and examples.

vip import media --help

Example command for importing an archived file available at the URL “https://www.example.com/uploads.tar.gz”:

vip @mytestsite.production import media "https://www.example.com/uploads.tar.gz"

During the import process, the CLI will display the real-time progress of the import. If desired, the real-time progress of an import can be hidden using the Ctrl + C key combination.

When all files in the archive have been processed, a status report for the import will be created within the current working directory. The exact path to this status report will be shown on screen.

Overwrite existing files during import

Including the --overwriteExistingFiles option (or -o ) during import will force the silent overwriting of existing files.

For example:

vip @mytestsite.production import media "https://www.example.com/uploads.tar.gz" --overwriteExistingFiles

Format error logs

If one or more files fail to import, an error log will be created on the local machine at the end of an import.

The file will be created within the path /Users/youruser/ with the file format media-import-<app_alias>-<date>.txt.

Error logs include a list of failed files and the reasons for their failure. Error logs are unable to record more than 250,000 file failures.

By default, import logs are created in .txt format. Add the --exportFileErrorsToJson option (or -E) to write the error logs to a .json file instead.

vip @mytestsite.production import media "https://www.example.com/uploads.tar.gz" -E

Abort an import

The vip import media abort command allows an import process to be aborted after the process has begun. All files imported prior to executing the abort command cannot be deleted.

vip @mytestsite.production import media abort

Check the status of an import

The status of the current or most recent import can be checked with the vip import media status command.

• If an import is currently running, the output will be for the currently running import, and the status command will continually check and report back on the status of the import process until it is in a finished state (COMPLETED, ABORTED, or FAILED).
• If an import is not currently running, the output will be for the import that ran most recently.

The Imported Files: field displays the number and percentage of files imported successfully.

If any files failed to import, a warning (⚠️) message will appear in the output, and the associated errors will be exported to an error log for review.

vip @mytestsite.production import media status
=============================================================
Checking the Media import status for your environment...
Imported Files: 95/100 - 95% ✓
=============================================================
Status: COMPLETED ✓ : The imported files should be visible on your App
App: mytestsite (PRODUCTION)
=============================================================

⚠️  5 file error(s) have been extracted

All errors have been exported to /Users/myuser/media-import-mytestsite-1642371503508.txt

The Status: field will display the overall import status with the following possible states:

Import status	Description
INITIALIZING	Contents of the archive file in the URL are downloading.
RUNNING	Files are being imported to the File System.
COMPLETING	The import is finishing up.
COMPLETED	The import is completed.
ABORTING	The import is responding to a vip import media abort request.
ABORTED	The import is aborted.
FAILED**	The import failed.

**The import can be in a FAILED status for a variety of reasons. Review status output, error logs, limitations, and the import checklist before attempting another media import.

Import states in the Audit Log

An audit trail of VIP-CLI commands that were executed to start or abort a media import, along with the timestamp of the commands and the VIP user that ran them, is displayed in the Audit Log panel of the VIP Dashboard.