Server API

Project Introduction

Environment Requirements

Dependencies	Version Requirements	Purpose
node	v18.19.0+	Runtime
mysql	5.7+	Database
redis	5.0+	Distributed Lock
ffmpeg	6.1+	Video Processing
yarn	latest	Package Management
git	latest	Version Control

Tech Stack

Tech Stack	Version	Purpose	Source Repository
Eggjs	3.17.3	Framework	-
Sequelize	6.33.0	Database ORM	-
Node Media Server	2.7.3	Media Server	-
AliyunOSS	7.18.1	Object Storage	-
Redis	4.6.10	Distributed Lock	-
Ffmpeg	6.1+	Video Processing	-
Minio	7.1.3	Object Storage	-
Aliyun SMS	3.1.1	SMS Service	-
Aliyun SMS	3.1.1	SMS Service	-
Tencent COS	2.12.0	Object Storage	-
UCloud US3	2.8.0	Object Storage	-
Nacos	2.1.1	Configuration Center	-
Socket.io	4.6.2	Real-time Communication	-

Project Structure

├── app/                    # Application core directory
│   ├── controller/         # Controllers
│   │   ├── admin/          # Admin backend controllers
│   │   ├── api/            # API interface controllers
│   │   ├── app/            # App backend controllers
│   │   └── index.js        # Controller export
│   ├── extend/             # Framework extensions
│   ├── middleware/         # Middleware
│   ├── model/              # Database models
│   ├── public/             # Static resources
│   ├── router.js           # Route configuration
│   ├── schedule/           # Scheduled tasks
│   ├── service/            # Business logic services
│   └── utils/              # Utility functions
├── build/                  # Build directory
│   └── nginx/              # Nginx configuration
├── config/                 # Configuration files
│   ├── config.default.js   # Default configuration
│   ├── config.local.js     # Local environment configuration
│   ├── config.prod.js      # Production environment configuration
│   └── plugin.js           # Plugin configuration
├── docs/                   # Documentation
├── logs/                   # Log files
├── migration/              # Database migrations
├── seeders/                # Database seeders
├── app.js                  # Application entry
├── docker-compose.yml      # Docker Compose configuration
├── Dockerfile              # Docker build file
├── nacos.js                # Nacos configuration script
├── .env.example            # Environment variables example
├── .eslintrc               # ESLint configuration
├── .gitignore              # Git ignore file
├── package.json            # Package dependencies
└── README.md               # Project README

QuickStart

Pull Code

$ git clone https://gitlab.com/h5ds/h5-video-server.git
$ cd h5-video-server

Configure Environment

$ cp .env.example .env

Modify the .env file:

# Database configuration
mysql.host=127.0.0.1
mysql.port=3306
mysql.database=video
mysql.username=root
mysql.password=123456

# Redis configuration
redis.host=127.0.0.1
redis.port=6379
redis.password=
redis.db=0

# Upload directory configuration
upload.dir=uploads
# CDN resource domain name
upload.resource=static

# Storage configuration
storage.type=local
# Local storage configuration
storage.local.path=./uploads
# Aliyun OSS configuration
storage.aliyun.region=oss-cn-hangzhou
storage.aliyun.accessKeyId=
storage.aliyun.accessKeySecret=
storage.aliyun.bucket=
storage.aliyun.endpoint=oss-cn-hangzhou.aliyuncs.com
storage.aliyun.baseUrl=

# Minio configuration
storage.minio.endPoint=127.0.0.1
storage.minio.port=9000
storage.minio.accessKey=
storage.minio.secretKey=
storage.minio.useSSL=false
storage.minio.bucket=
storage.minio.baseUrl=

# Tencent COS configuration
storage.cos.region=ap-guangzhou
storage.cos.secretId=
storage.cos.secretKey=
storage.cos.bucket=
storage.cos.baseUrl=

# UCloud US3 configuration
storage.us3.publicKey=
storage.us3.privateKey=
storage.us3.bucket=
storage.us3.endpoint=us3.hk.ufileos.com
storage.us3.region=hk
storage.us3.baseUrl=

# Render service configuration
render.host=http://localhost:3333

# Video convert service configuration
convert.host=http://localhost:3000

# SMS service configuration
# Aliyun SMS configuration
aliyunSms.accessKeyId=
ali yunSms.accessKeySecret=
ali yunSms.signName=
ali yunSms.templateCode=

# Socket.IO configuration
io.enable=false
io.port=3001

# Nacos configuration center
nacos.serverList=
nacos.client.namespace=
nacos.client.serviceName=
nacos.client.groupName=DEFAULT_GROUP
nacos.client.username=
nacos.client.password=

Install Dependencies

$ yarn install

Initialize Database

# Initialize database structure
$ yarn init-server

Development Environment Deployment

# Debug mode (with breakpoints)
$ yarn debug

# Development mode (hot reload)
$ yarn dev

# Production mode
$ yarn start

# Stop service
$ yarn stop

# Restart service
$ yarn restart

Test Interface

API documentation address：Link Us

Database Design

Database Design Document

Online Deployment

Double Port Mode (Recommended)

# Start port 8881
$ PORT=8881 yarn start

# Start port 8882
$ PORT=8882 yarn start

Then configure load balancing through Nginx.

Single Port Mode

# Start service
# The system will use the configuration in config/config.local.js by default (refer to the official documentation to set environment parameters if you need to specify a configuration file)
# The system will listen to http://0.0.0.0:{port}/, and can provide services externally at http://server-ip:{port}
$ yarn start

# Stop service
$ yarn stop

# Restart service
$ yarn restart

Nginx Configuration

[Hot Deployment Mode] (Recommended), supports single-machine non-downtime updates. If you need to directly proxy the front-end page with Nginx and need to support HTTPS, the following configuration is recommended:

Note: The mime.types file introduced by Nginx must include the wasm type:

application/wasm                                 wasm;

# For single-machine hot deployment
upstream us-fvideo {
    server localhost:8882;
    server localhost:8881;
}

server {
    listen 443;
    server_name video.h5ds.com;
    ssl on;
    ssl_certificate /opt/app/letsencrypt/h5ds.com/fullchain.pem;
    ssl_certificate_key /opt/app/letsencrypt/h5ds.com/privkey.pem;
    ssl_session_timeout 5m;
    client_max_body_size 1000M;

    # Proxy WeChat related (cross-domain address)
    location ~ ^/cgi-bin {
        proxy_pass https://mp.weixin.qq.com;
    }

    # Proxy (cross-domain address)
    location ~ ^/(video/|static/|uploads/) {
        proxy_pass https://cdn.h5ds.com;
    }

    # Front-end packaged pages and resource directory
    root "/www/video/web";

    # Front-end - Official website pages
    location / {
        index index.html;
        try_files $uri $uri/ /index.html;
        if ($request_filename ~* .*\.html$) {
           add_header Cache-Control "no-cache, no-store";
        }
    }

    # H5 editor resource directory
    location ~ ^/(editor$|editor/) {
        add_header Cross-Origin-Embedder-Policy "require-corp" always;
        add_header Cross-Origin-Opener-Policy "same-origin" always;
        try_files /editor.html =404;
    }

    # Front-end - Management backend
    location ~ ^/(admin$|admin/) {
        try_files /admin.html =404;
    }

    # Resource path sets cross-domain headers (otherwise cross-domain errors will occur in the editor)
    location ~ ^/assets/ {
        add_header Cross-Origin-Embedder-Policy "require-corp" always;
        add_header Cross-Origin-Opener-Policy "same-origin" always;
        index index.html;
        try_files $uri $uri/ /index.html;
    }

    # Server API interface service
    location ~ ^/(api$|api/|video/) {
        # Large file upload limit
        client_max_body_size 1000M;
        # Define buffers, necessary for proper communication to prevent 502s
        proxy_buffer_size 128k;
        proxy_buffers 4 256k;
        proxy_busy_buffers_size 256k;
        proxy_connect_timeout 1200;
        proxy_read_timeout 1200;
        proxy_send_timeout 1200;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header REMOTE-HOST $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        # Proxy to the IP and port that the server listens on when starting, note that the value of proxy_pass must not end with /
	    #proxy_pass http://172.31.195.253:8882;
        proxy_pass http://us-video;
    }

    location ~ .*\.(gif|jpg|jpeg|png|bmp|swf)$ {
        expires 30d;
    }

    location ~ .*\.(js|css)?$ {
        expires 12h;
    }

    # Define error page codes, if corresponding error page codes appear, forward to them.
    error_page 404  /404.html;
}

server {
    listen 80;
    server_name video.h5ds.com localhost;
    # For Let's Encrypt certificate verification
    
    location / {
        # Redirect HTTP domain requests to HTTPS
        return 301 https://$host$request_uri;
    }
}

Docker Build and Use

The container includes: front-end code + server service + nginx service

Build and Test

1. Put the built front-end code into the project's app/public/web directory

2. Modify the nginx configuration file: build/nginx/conf.d/video.conf

3. Modify the server configuration file: .env file

4. Build & test the image:

# Delete old container
$ docker stop video
$ docker rm video

# Build image
$ docker build -t video .

# Test image
$ docker run -it --name video -v $PWD/.env:/www/server/.env -v $PWD/app:/www/server/app -p 80:80 -p:443:443 -p 8882:8882 video

# Description
    -v $PWD/.env:/www/server/.env  Configuration file
    -v $PWD/app:/www/server/app    Business code
    -p 80:80   Map port 80
    -p 443:443 Map port 443

# Push to remote image repository
$ docker tag render registry.cn-hangzhou.aliyuncs.com/h5ds/video:latest
$ docker push registry.cn-hangzhou.aliyuncs.com/h5ds/video:latest

# Clean up <none> tag images
$ docker image prune -f

Container Deployment

Integration with concat service requires ensuring that the server/app/public/video directory (resource upload, generation, access directory) is consistent and accessible in both server and concat services.

Assuming the overall directory structure is:

server/
      .env    # Server service configuration
concat/
      .env    # Concat service configuration
video/        # Resource upload, generation, access directory, mounted to the same location in server and concat containers respectively

Run the containers:

# Run server container
$ docker run -it --name video -v $PWD/server/.env:/www/server/.env -v $PWD/video:/www/server/app/public/video/ -p 80:80 -p:443:443 video

# Run concat container
$ docker run -it --name concat -v $PWD/concat/.env:/www/concat/.env -v $PWD/video:/www/server/app/public/video/ concat

# Note: In concat's .env file

# Description
    -v $PWD/*/.env:/www/*/.env                     Configuration files
    -v $PWD/video:/www/server/app/public/video/    Business code
    -p 80:80   Map port 80
    -p 443:443 Map port 443

Project Configuration Instructions

1. [.env] Local Configuration File Mode

In this mode, there are two configuration methods (Note: Keep only one configuration file of the following two methods, otherwise configuration may not take effect):

Method 1: Do not distinguish between running environments
    Only copy one .env configuration file, at this time when starting the project with yarn debug or yarn dev or yarn start, the configuration of .env will be applied

$ cp .env.example .env

Method 2: Distinguish between running environments (currently the system supports local and prod environments)
    Copy two configuration files .env.local and .env.prod, and modify the configuration values in them.
    a. When users start the project with yarn debug or yarn dev, the configuration of .env.local will be applied
    b. When users start the project with yarn start, the configuration of .env.prod will be applied

$ cp .env.example .env.local
$ cp .env.example .env.prod

Note: Currently, the project does not provide a test environment configuration mode. Users can refer to the current environment configuration and expand more environment configurations according to their project conditions.

Configuration file content example, refer to .env.example file

2. [Nacos] Remote Configuration Center Mode (Configuration Center & Service Registration & Service Discovery Call)

Configuration center, supports obtaining configuration data through username+password

1. Delete the .env file in the current directory

2. Modify the config/config.nacos.js configuration file, set the configuration center connection information

3. Start in local mode
$ yarn nacos-dev

4. Start in prod mode
$ yarn nacos-start

Implementation principle description:

1. Modify the config/config.nacos.js configuration file, set the configuration center connection information

2. Write the configuration center content to the local .env configuration file by calling node nacos.js

3. Write the configuration from the local cache file to the environment variable process.env by calling the following statement in config.default.js/config.local.js/config.prod.js files:
// Load .env configuration into process.env environment variables
require("dotenv").config({ path: ".env" });

4. Adjust the scripts startup script in the project's package.json file
node nacos.js && ~~~

5. Modify the configuration file to take values from environment variables, for example:
host: process.env["sequelize.host"] || "",
port: process.env["sequelize.port"] || ""

Service Registration

1. Modify config/plugin.js, enable the plugin

    exports.nacos = {
    enable: true,
    package: "eggjs-nacos"
    };

2. Modify the configuration related to nacos in the env configuration file

    #nacos service registration and discovery configuration
    nacos.serverList=nacos.h5ds.com
    nacos.client.namespace=dev
    nacos.client.serviceName=h5ds-v7
    nacos.client.groupName=DEFAULT_GROUP
    nacos.client.username=
    nacos.client.password=

Service Call

Reference code
async serviceInvoke() {
    const result = await this.ctx.nacos.h5ds.request("/api/v1/test"); // Default GET request
    if (result.status !== 200) throw Error("Error ...");
    this.ctx.body = result.data;
    return;
}

Common Scripts

• Use F5 or yarn debug [Recommended] to start the development environment and enable breakpoint debugging mode.
• Use yarn dev Development environment startup script.
• Use yarn start Production environment startup script.
• Use yarn init-server Initialize database structure and data (only need to execute once at the beginning)
• Use npx sequelize migration:generate --name=init-users Create database migration file
• Use yarn migrate:create --name=init-users Create database migration file
• Use yarn migrate Execute data migration, use config/config.local.js configuration file
• Use yarn migrate-prod Execute data migration, use config/config.prod.js configuration file
• Use npx sequelize db:migrate:undo Undo the most recent data migration

Command Line Tool Artisan

Usage:

# View all commands
$ yarn artisan

# View help for specific command (init)
$ yarn artisan -- init -h

# View help for specific command (material)
$ yarn artisan -- material -h

# Test command
$ yarn artisan -- test -x=1 --type=2 a b

Common Commands After Enabling Nacos

• Use F5 or yarn nacos-debug [Recommended] to start the development environment and enable breakpoint debugging mode.
• Use yarn nacos-dev Local development mode startup script.
• Use yarn nacos-start Production environment startup script.
• Use yarn nacos-migrate Execute data migration, use config/config.local.js configuration file
• Use yarn nacos-migrate-prod Execute data migration, use prod namespace data from nacos, and use config/config.prod.js configuration file

QA

1. Error when executing yarn artisan on mac/linux: env: node\r: No such file or directory

   Reason: Because the line ending sequence used by the egg-artisan package is CRLF (windows default), this sequence will recognize \r, \t and other characters as strings in mac/linux, causing the file not found error

   Solution: Should change node_modules/egg-artisan/*.js to LF sequence