User Tools

Site Tools


en:feature_autoimport

Auto Import

The scheme of work is as follows:

  • The request to the cdn.domain.com/path/to/file.mp4 file comes to the CDN.
  • If the file is in the CDN, then it is given to the client
  • If there is no such file in the CDN, the user will be redirected to a special URL (url4fail), which usually points to the storage - storage.domain.com/path/to/file.mp4. The requested file will be in the queue to be added and after a while, it will be downloaded to the CDN. After that, requests on this file will be served by the CDN.

It's possible to make a redirect to a secured link to the storage (see below). You can also change the link to which it's needed to redirect (by default, the file name in the CDN is added to the and url4fail link ) if the file path in the file in CDN does not match the file path on the file in storage.

If a large amount of traffic is expected, then with such scheme it is better to pre-import the most popular files so a lot of traffic won't go to the Storage. Or you can change the links to the files in parts, or switch on the traffic for a short period of time in order to fill the queue for adding and switch off the traffic. Then wait when the files in queue will be downloaded to CDN and then again switch the traffic to CDN.

If you want to enable a secure link in NGINX, you will need the module http://nginx.org/en/docs/http/ngx_http_secure_link_module.html . An example of a host configuration to enable a secure link :

server {
  listen 192.168.0.1:80 ;
  server_name storage4cdn.example.com;
  error_log  /dev/null;
  access_log off;
  location ~ /\.ht { deny all; }
  
  location /s/ {
    secure_link $arg_md5,$arg_time;
    secure_link_md5 "SECRET_SALT$uri$secure_link_expires$remote_addr";

    if ($secure_link = "") {
        return 403;
    }

    if ($secure_link = "0") {
        return 410;
    }
    alias /home/user/example.com/contents/videos/;
  }
}

SECRET_SALT secret word with any set of characters which you have to inform Support Team of the CDN about. When requesting a file which is not in the CDN, the user will be redirected to this kind of link

http://storage4cdn.example.com/s/1000/1111/1111.mp4?md5=R-f0UltrpbTX9xVzS-3z4g&time=1489617943

How to check delivery through secure virtualhost

You can generate the link for the file /1000/1111/1111.mp4 on the domain storage4cdn.example.com

#!/bin/sh

DOMAIN='storage4cdn.example.com'
SECRET_SALT='some_salt'
EXPIRES=`date +%s`; EXPIRES=$(($EXPIRES+3600))  # lifetime of the link
REMOTE_ADDR='1.2.3.4' # IP from which you will check the link
FILE='/s/1000/1111/1111.mp4'
KEY=`echo -n "$SECRET_SALT$FILE$EXPIRES$REMOTE_ADDR" | openssl md5 -binary | openssl base64 | tr +/ -_ | tr -d =`
echo "$DOMAIN$FILE?md5=$KEY&time=$EXPIRES"
en/feature_autoimport.txt · Last modified: 2018/11/28 15:29 by panda