How to Install RustDesk Remote Desktop Server on Ubuntu

Introduction

RustDesk is an open-source remote desktop solution that gives you full control of your remote connections through a self-hosted private server. It's an alternative to closed-source applications such as TeamViewer and AnyDesk with a lightweight, privacy-centric interface.

This guide describes how you can install the RustDesk remote desktop server on Ubuntu, and connect remote devices using a RustDesk client application.

Prerequisites

Before your begin, make sure you:

• Deploy a Ubuntu server on Vultr.

RustDesk is lightweight, and you can install it on an existing Ubuntu server running docker.

• Setup a subdomain A record that points to the server.

A subdomain masks your direct public server IP Address protecting it from potential attacks.

• Install Docker compose.
• Use SSH to access the server as a non-root sudo user.

1. Install RustDesk

1. Create a new directory to store RustDesk docker files.

    $ sudo mkdir -p /opt/rustdesk
   

2. Switch to the directory.

    $ cd sudo /opt/rustdesk
   

3. Create a new rustdesk.yml file.

    $ sudo touch rustdesk.yml
   

4. Edit the file using a text editor like Nano.

    $ sudo nano rustdesk.yml
   

5. Add the following configurations to the file.

    version: '3'
   
      networks:
        rustdesk-net:
        external: false
   
    services:
       hbbs:
         container_name: hbbs
       ports:
         - 21115:21115
         - 21116:21116
         - 21116:21116/udp
         - 21118:21118
       image: rustdesk/rustdesk-server:latest
       command: hbbs -r 127.0.0.1:21117 -k _
       volumes:
          - ./hbbs:/root
       networks:
          - rustdesk-net
       depends_on:
          - hbbr
       restart: unless-stopped
   
       hbbr:
          container_name: hbbr
       ports:
          - 21117:21117
          - 21119:21119
       image: rustdesk/rustdesk-server:latest
       command: hbbr -k _
       volumes:
          - ./hbbr:/root
       networks:
          - rustdesk-net
       restart: unless-stopped
   

   Save and close the file.

6. Start the RustDesk server containers.

    $ sudo docker-compose -f rustdesk.yml up -d
   

7. Verify that RustDesk hbbs, hbbr containers are up and running.

    $ sudo docker ps
   

2. Secure the Server

While using RustDesk directly in a local development environment is safe, this would expose your backend port in production. To safely use RustDesk, set up Nginx as a reverse proxy to serve requests through a subdomain to the backend hbbs and hbbr ports.

1. First, allow Nginx to communicate on HTTP 80 through the default firewall (UFW).

    $ sudo ufw allow 80/tcp
   

2. Reload Firewall rules.

    $ sudo ufw reload
   

3. Create a new Nginx configuration file.

    $ sudo touch /etc/nginx/conf.d/rustdesk.example.conf
   

4. Edit the file.

    $ sudo nano /etc/nginx/conf.d/rustdesk.example.conf
   

5. Add the following configuration to the file.

    server {
        listen 80;
        listen [::]:80;
   
    # Set your subdomain
   
        server_name rustdesk.example.com;
   
    # Proxy Requests to the RustDesk host port
   
        location / {
          proxy_pass http://127.0.0.1:21117;
          proxy_set_header Host $host;
          proxy_set_header X-Real-IP $remote_addr;
        }
    }
   

   Save and close the file.

6. Test the Nginx configuration for errors.

    $ sudo nginx -t
   

7. Restart Nginx.

    $ sudo systemctl restart nginx
   

View RustDesk Server Encryption Keys

Based on your Docker compose configuration, the RustDesk server encrypts all connections based on your public key available in the hbbs directory. Any RustDesk client without the key cannot connect through the relay server.

1. View and copy your server public key.

    $ cat /opt/rustdesk/hbbs/id_ed25519.pub
   

2. Your Output should look like the one below.

    WfgvfV002mcFRevj205Fd+3vEEEWpeHlcL3xAXnAxJ4=
   

3. Configure the RustDesk Client

1. On a client machine, download the RustDesk client application from the official website.
2. Install the application on the client computer.
3. Open the RustDesk application.
4. Click the settings 站ｮ button next to your connection ID.
5. Select ID/Relay Server from the list of options.
6. Enter your server's domain name in the ID Server field.
7. Enter the server public key in the Key field.

8. Click OK to save changes and establish a connection to your RustDesk server.

9. Copy the remote computer ID and password, or click Settings, navigate to Change ID and set it to your preferred value.

4. Test the Server

1. On your local computer, download and install the RustDesk client.
2. Open the application, and navigate to ID/Relay Server.
3. Enter your RustDesk server's domain name in the ID Server: field.
4. Enter the server public key.
5. Click OK to save changes.
6. Enter your remote machine ID in the Enter Remote ID field.
7. Click Connect to establish a connection through your RustDesk server to the remote machine.
8. Your remote computer's desktop displays. The connection should feel fast and encrypted as your data routes through a single private RustDesk server.

Conclusion

You have successfully installed RustDesk server on Ubuntu and established a connection to a remote machine. The client application is available for Windows, Linux, macOS, Android, and iOS devices. For more information, please visit the official RustDesk documentation.