Metascan Anti-malware Protection Add-on

Overview

Cloudinary is a cloud-based service that provides an end-to-end image management solution including uploads, storage, manipulations, optimizations and delivery.

Cloudinary offers a very rich set of image uploading, manipulation and media library management capabilities. Cloudinary allows you to upload images to the cloud, manipulate them on-the-fly and deliver them to your users optimized and cached via a fast CDN.

OPSWAT's Metascan offers a server application that enables detecting and preventing advanced threats by incorporating multi-scanning, data sanitization technology, and controlled data workflows. Cloudinary provides an add-on for using Metascan's anti-malware solution, fully integrated into Cloudinary's image management and manipulation pipeline. The Metascan add-on leverages four antivirus engines simultaneously: ESET, AVG, ClamWin and Norman to prevent viruses and malwares from creeping into your websites or mobile applications.

With Metascan's anti-malware add-on, you can extend Cloudinary's powerful cloud-based image media library and delivery capabilities with automatically scanning of user uploaded photos and documents. When using the Metascan add-on, user uploaded files are automatically scanned, making sure that no virus or malicious code is delivered to your web and mobile viewers.

Automatic file scanning flow

The following list describes the flow of uploading and displaying moderated images using Cloudinary and the Metascan anti-malware add-on:

Image upload
- Your users upload an image, or a file of any kind, to Cloudinary through your application.
- The uploaded images are set to a 'pending' status, with short term CDN caching.
- You can decide whether to display the pending image or wait for its approval.
Image moderation
- The uploaded image is sent to Metascan for asynchronous moderation in the background.
- The image is either approved or rejected by Metascan's anti-malware add-on.
- An optional notification callback is sent to your application with the virus scanning result.
- If the image is approved, its cache settings are modified to be long-term.
- A rejected image is removed and the original rejected image is move to a secondary backup repository.
Manual override
- Pending, approved and rejected images can be listed programatically using Cloudinary's API or interactively using our online Media Library Web interface.
- You can manually override the automatic moderation using the API or Media Library.

Request file scanning

Before you start, if you haven't done so already, please sign-up to a free Cloudinary account. After signing up, you can try the Metascan anti-malware protection add-on for free and later on subscribe to an Metascan add-on plan that best matches your usage requirements.

To simplify the add-on integration, you can use Cloudinary's client libraries for: Ruby on Rails, Python & Django, PHP, Node.js, jQuery, AngularJS, Java, .NET, iOS, Android and Scala.

In order to request moderation of uploaded images, simply set the 'moderation' upload API parameter to 'metascan':

Ruby PHP Python Node.js Java All

Ruby:

Cloudinary::Uploader.upload("local_file.jpg", 
  :moderation => "metascan")

PHP:

\Cloudinary\Uploader::upload("local_file.jpg", 
  array("moderation" => "metascan"));

Python:

cloudinary.uploader.upload("local_file.jpg",
  moderation = "metascan")

Node.js:

cloudinary.uploader.upload("local_file.jpg", 
  function(result) { console.log(result); }, 
  { moderation: "metascan" });

Java:

cloudinary.uploader().upload("local_file.jpg", 
  ObjectUtils.asMap("moderation", "metascan"));

Same goes for non-image raw files. You need to set the resource_type parameter to raw when calling the upload API method:

Ruby PHP Python Node.js Java All

Ruby:

Cloudinary::Uploader.upload("virus.txt", :resource_type => :raw,
  :moderation => "metascan")

PHP:

\Cloudinary\Uploader::upload("virus.txt", 
  array("resource_type" => "raw", "moderation" => "metascan"));

Python:

cloudinary.uploader.upload("virus.txt", resource_type = "raw",
  moderation = "metascan")

Node.js:

cloudinary.uploader.upload("virus.txt", 
  function(result) { console.log(result); }, 
  { moderation: "metascan", resource_type: "raw" });

Java:

cloudinary.uploader().upload("virus.txt",  
  ObjectUtils.asMap("resource_type", "raw", "moderation", "metascan"));

The uploaded file is available for delivery based on the randomly assigned public ID with short-term caching of 10 minutes. Image moderation by the Metascan antirus add-on is performed asynchronously and should be completed within up to a few minutes.

The following snippet shows the response of the upload API call that signifies that the Metascan moderation is in the 'pending' status.

Ruby PHP Python Node.js Java All

Ruby:

{
  "public_id" => "iswyvngmi8avzvomdpfc",
  "version" => 1393751993,
  "url" =>
    "http://res.cloudinary.com/demo/image/upload/v1393751993/iswyvngmi8avzvomdpfc.jpg",
  "moderation" => [{"status"=>"pending", "kind"=>"metascan"}],
  ...        
}

PHP:

array(14) {
  ["public_id"]=>
  string(20) "iswyvngmi8avzvomdpfc"
  ["version"]=>
  int(1393751993)
  ["url"]=>
  string(80) "http://res.cloudinary.com/demo/image/upload/v1393751993/iswyvngmi8avzvomdpfc.jpg"
  ["moderation"]=>
  array(1) {
    [0]=>
    array(2) {
      ["status"]=>
      string(7) "pending"
      ["kind"]=>
      string(9) "metascan"
    }
  }
  ...
}

Python:

{ 
  u'public_id': u'iswyvngmi8avzvomdpfc',
  u'version': 1393751993,
  u'url': 
    u'http://res.cloudinary.com/demo/image/upload/v1393751993/iswyvngmi8avzvomdpfc.jpg',
  u'moderation': [{ u'kind': u'metascan', u'status': u'pending'}],
  ...
}

Node.js:

{ 
  public_id: 'iswyvngmi8avzvomdpfc',
  version: 1393751993,
  url: 
    'http://res.cloudinary.com/demo/image/upload/v1393751993/iswyvngmi8avzvomdpfc.jpg',
  moderation': [{ 'kind': 'metascan', 'status': 'pending'}],
  ...
}

Java:

{
  "public_id": "iswyvngmi8avzvomdpfc",
  "version": 1393751993,
  "url": "http:\/\/res.cloudinary.com\/demo\/image\/upload\/v1393751993\/iswyvngmi8avzvomdpfc.jpg",
  "moderation": [{ "status": "pending", "kind": "metascan" }],
  ...
}

Status notification

Due to the fact that the Metascan add-on moderates images asynchronously, you might want to get notified when the scanning process is completed.

When calling the upload API with Metascan file scanning, you can set the 'notification_url' to a public HTTP or HTTPS URL of your online web application. Cloudinary sends a POST request to the specified endpoint when Metascan scanning is completed.

Ruby PHP Python Node.js Java All

Ruby:

Cloudinary::Uploader.upload("local_file.jpg", 
  :moderation => "metascan",
  :notification_url => "http://mysite.example.com/hooks")

PHP:

\Cloudinary\Uploader::upload("local_file.jpg", 
  array(
    "moderation" => "metascan",
    "notification_url" => "http://mysite.example.com/hooks"));

Python:

cloudinary.uploader.upload("local_file.jpg",
  moderation = "metascan",
  notification_url = "http://mysite.example.com/hooks")

Node.js:

cloudinary.uploader.upload("local_file.jpg", 
  function(result) { console.log(result); }, 
  { 
    moderation: "metascan",
    notification_url: "http://mysite.example.com/hooks" 
  });

Java:

cloudinary.uploader().upload("local_file.jpg",
  ObjectUtils.asMap("moderation", "metascan",
                    "notification_url", "http://mysite.example.com/hooks"));

The following JSON snippets are examples of POST requests sent to the notification URL when moderation is completed. The 'moderation_status' value in this case can be either 'approved' or 'rejected'

{
  "moderation_response": "approved",
  "moderation_status": "approved",
  "moderation_kind": "metascan",
  "moderation_updated_at": "2014-03-02T09:33:43Z",
  "public_id": "l8yvniicjhmyxuuqqyqy",
  "uploaded_at": "2014-03-02T09:33:42Z",
  "version": 1393752822,
  "url": 
    "http://res.cloudinary.com/demo/image/upload/v1393752822/l8yvniicjhmyxuuqqyqy.jpg",
  "secure_url": 
    "https://res.cloudinary.com/demo/image/upload/v1393752822/l8yvniicjhmyxuuqqyqy.jpg",
  "etag": "83340520d28b704ca4f4b019effb33dc",
  "notification_type": "moderation" 
}

{
  "moderation_response": "rejected",
  "moderation_status": "rejected",
  "moderation_kind": "metascan",
  "moderation_updated_at": "2014-03-02T20:47:48Z",
  "public_id": "nuzn4riqxhhzfyfljjxv",
  "uploaded_at": "2014-03-02T20:47:47Z",
  "version": 1393793267,
  "url":
    "http://res.cloudinary.com/demo/image/upload/v1393793267/nuzn4riqxhhzfyfljjxv.jpg",
  "secure_url":
    "https://res.cloudinary.com/demo/image/upload/v1393793267/nuzn4riqxhhzfyfljjxv.jpg",
  "etag":"06778590d96907b60b5fa83795e7df3b",
  "notification_type":"moderation"
}

The JSON content is signed using the API Secret of your Cloudinary account. For more details regarding Cloudinary's notifications and webhooks, see this blog post.

Query status

As an alternative to receiving a webhook when image moderation is completed, you can poll Cloudinary's Admin API for checking the image moderation status of previously uploaded image:

Ruby PHP Python Node.js Java All

Ruby:

Cloudinary::Api.resource('l8yvniicjhmyxuuqqyqy')

PHP:

$api = new \Cloudinary\Api();
$api->resource("l8yvniicjhmyxuuqqyqy");

Python:

cloudinary.api.resource("l8yvniicjhmyxuuqqyqy")

Node.js:

cloudinary.api.resource('l8yvniicjhmyxuuqqyqy', 
  function(result)  { console.log(result) });

Java:

cloudinary.api().resource("l8yvniicjhmyxuuqqyqy", 
  ObjectUtils.emptyMap());

The following JSON snippet is the response of the Admin API resource details of an image being moderated. As you can see, the moderation status is set to 'approved'.

{ 
  "public_id": "l8yvniicjhmyxuuqqyqy", 
  "format": "jpg", 
  "version": 1393752822, 
  "resource_type": "image", 
  "type": "upload", 
  "created_at": "2014-03-02T09:33:42Z", 
  "bytes": 120253, 
  "width": 864, 
  "height": 576, 
  "backup": true, 
  "url":
    "http://res.cloudinary.com/demo/image/upload/v1393752822/l8yvniicjhmyxuuqqyqy.jpg", 
  "secure_url": 
    "https://res.cloudinary.com/demo/image/upload/v1393752822/l8yvniicjhmyxuuqqyqy.jpg", 

  "moderation":
    [
      {
        "response": {
          "result_code": 0,
          "result": "Clean",
          "engine_results": [
            {
              "engine": "AVG",
              "result_code": 0,
              "threat_found": ""
            },
            {
              "engine": "ClamWin",
              "result_code": 0,
              "threat_found": ""
            },
            {
              "engine": "ESET",
              "result_code": 0,
              "threat_found": ""
            },
            {
              "engine": "Norman",
              "result_code": 0,
              "threat_found": ""
            }
          ]
        },
        "status": "approved",
        "kind": "metascan",
        "updated_at": "2014-06-19T13:20:19Z"
      }
    ]
   "next_cursor": "699718787ae2aa6423bd1946864e8768", 
   "derived": []
 }

The example below queries the Metascan moderation status of an uploaded raw text file:

Ruby PHP Python Node.js Java All

Ruby:

Cloudinary::Api.resource('pxt3lqcn009irvwaokia.txt', :resource_type => "raw")

PHP:

$api = new \Cloudinary\Api();
$api->resource("pxt3lqcn009irvwaokia.txt", array("resource_type" => "raw"));

Python:

cloudinary.api.resource("pxt3lqcn009irvwaokia.txt", resource_type = "raw")

Node.js:

cloudinary.api.resource('pxt3lqcn009irvwaokia.txt', 
  function(result)  { console.log(result) }, { resource_type: "raw" });

Java:

cloudinary.api().resource("l8yvniicjhmyxuuqqyqy", 
  ObjectUtils.asMap("resource_type", "raw"));

This file contains a virus signature and therefore it is rejected. The JSON response includes details regarding the detected virus by all antivirus engines used by the Metascan add-on.

{
  "public_id": "pxt3lqcn009irvwaokia.txt",
  "version": 1403184472,
  "resource_type": "raw",
  "type": "upload",
  "placeholder": true,
  "created_at": "2014-06-19T13:27:52Z",
  "bytes": 0,
  "backup": true,
  "url": "http://res.cloudinary.com/demo/raw/upload/v1403184472/pxt3lqcn009irvwaokia.txt",
  "secure_url": "https://res.cloudinary.com/demo/raw/upload/v1403184472/pxt3lqcn009irvwaokia.txt",
  "moderation": [
    {
      "response": {
        "result_code": 1,
        "result": "Infected",
        "engine_results": [
          {
            "engine": "AVG",
            "result_code": 1,
            "threat_found": "EICAR_Test"
          },
          {
            "engine": "ClamWin",
            "result_code": 1,
            "threat_found": "Eicar-Test-Signature"
          },
          {
            "engine": "ESET",
            "result_code": 1,
            "threat_found": "Eicar test file"
          },
          {
            "engine": "Norman",
            "result_code": 1,
            "threat_found": "doslegacy/EICAR_Test_file_not_a_virus!"
          }
        ]
      },
      "status": "rejected",
      "kind": "metascan",
      "updated_at": "2014-06-19T13:27:53Z"
    }
  ],
  "next_cursor": "59d9603fd8b375e54c295a8ec41ad4ce",
  "derived": [

  ]
}

Anti-malware moderation queue

Cloudinary's Admin API can be used to list the queue of all moderated images. You can list all three queues of pending, approved and rejected images by specifying the second parameter of the 'resources_by_moderation' API method.

Ruby PHP Python Node.js Java All

Ruby:

Cloudinary::Api.resources_by_moderation("metascan", "pending")

PHP:

$api = new \Cloudinary\Api();
$api->resources_by_moderation("metascan", "pending");

Python:

cloudinary.api.resources_by_moderation("metascan", "pending")

Node.js:

cloudinary.api.resources_by_moderation('metascan', 'pending', 
  function(result)  { console.log(result) });

Java:

cloudinary.api().resourcesByModeration("metascan", "pending", 
  ObjectUtils.emptyMap());

{
 "resources"=>
  [{
    "public_id": "q7vcvrfjm9mj4bfp3qc8",
    "format": "jpg",
    "version": 1393794403,
    "resource_type": "image",
    "type": "upload",
    "created_at": "2014-03-02T21:06:43Z",
    "bytes": 120253,
    "width": 864,
    "height": 576,
    "backup": true,
    "url": 
     "http://res.cloudinary.com/demo/image/upload/v1393794403/q7vcvrfjm9mj4bfp3qc8.jpg",
    "secure_url": 
     "https://res.cloudinary.com/demo/image/upload/v1393794403/q7vcvrfjm9mj4bfp3qc8.jpg"
   },
   {
    "public_id": "zp4fgdbabhlwwa7bxu84",
    "format": "jpg",
    "version": 1393794399,
    "resource_type": "image",
    "type": "upload",
    "created_at": "2014-03-02T21:06:39Z",
    "bytes": 120253,
    "width": 864,
    "height": 576,
    "backup": true,
    "url": 
     "http://res.cloudinary.com/demo/image/upload/v1393794399/zp4fgdbabhlwwa7bxu84.jpg",
    "secure_url": 
     "https://res.cloudinary.com/demo/image/upload/v1393794399/zp4fgdbabhlwwa7bxu84.jpg"
   }
 ]
}

Listing the queue of images rejected by Metascan image scanning:

Ruby PHP Python Node.js Java All

Ruby:

Cloudinary::Api.resources_by_moderation("metascan", "rejected")

PHP:

$api = new \Cloudinary\Api();
$api->resources_by_moderation("metascan", "rejected");

Python:

cloudinary.api.resources_by_moderation("metascan", "rejected")

Node.js:

cloudinary.api.resources_by_moderation('metascan', 'rejected', 
  function(result)  { console.log(result) });

Java:

cloudinary.api().resourcesByModeration("metascan", "rejected", 
  ObjectUtils.emptyMap());

Listing the queue of images approved by Metascan image scanning:

Ruby PHP Python Node.js Java All

Ruby:

Cloudinary::Api.resources_by_moderation("metascan", "approved")

PHP:

$api = new \Cloudinary\Api();
$api->resources_by_moderation("metascan", "approved");

Python:

cloudinary.api.resources_by_moderation("metascan", "approved")

Node.js:

cloudinary.api.resources_by_moderation('metascan', 'approved', 
  function(result)  { console.log(result) });

Java:

cloudinary.api().resourcesByModeration("metascan", "approved", 
  ObjectUtils.emptyMap());

Manual override

While the automatic virus scanning of the Metascan add-on is very accurate, in some cases you may want to manually override the moderation decision. You can either approve a previously rejected image or reject an approved one.

One way to manually override the moderation result is using Cloudinary's Media Library Web interface.

As you can see in the following screenshot, you can click on the 'Moderation Queue' link, select 'Metascan' and then the specific queue of images. In this case, we list the queue of images rejected by Metascan. You can then click on the green Approve button to revert the decision and recover the original rejected image.

As the following screenshot illustrates, the same goes for the list of approved images. You can click on the red Reject button to revert the decision and prevent a certain image from being publicly available to your users.

Alternatively to using the Web interface, you can use Cloudinary's Admin API to manually override the moderation result. The following sample code uses the 'update' API method while specifying a public ID of a moderated image and setting the 'moderation_status' parameter to either the 'approved' or the 'rejected' status.

Ruby PHP Python Node.js Java All

Ruby:

Cloudinary::Api.update("hwepb67oxzh4lrigssld", 
  :moderation_status => "approved")

PHP:

$api = new \Cloudinary\Api();
$api->update("hwepb67oxzh4lrigssld", 
  array("moderation_status" => "approved"));

Python:

cloudinary.api.update("hwepb67oxzh4lrigssld",
  moderation_status = "approved")

Node.js:

cloudinary.api.update("hwepb67oxzh4lrigssld", 
  function(result) { console.log(result); }, 
  { moderation_status: "approved" });

Java:

cloudinary.api().update("hwepb67oxzh4lrigssld", 
  ObjectUtils.asMap("moderation_status", "approved"));

Ruby PHP Python Node.js Java All

Ruby:

Cloudinary::Api.update("hwepb67oxzh4lrigssld", 
  :moderation_status => "rejected")

PHP:

$api = new \Cloudinary\Api();
$api->update("hwepb67oxzh4lrigssld", 
  array("moderation_status" => "rejected"));

Python:

cloudinary.api.update("hwepb67oxzh4lrigssld",
  moderation_status = "rejected")

Node.js:

cloudinary.api.update("hwepb67oxzh4lrigssld", 
  function(result) { console.log(result); }, 
  { moderation_status: "rejected" });

Java:

cloudinary.api().update("hwepb67oxzh4lrigssld", 
  ObjectUtils.asMap("moderation_status", "rejected"));