Deploy Weaviate to EC2 using AWS CDK for TypeScript

Table of Contents

Introduction #

In the world of databases, vector systems like Weaviate are making significant strides. They’re robust, configurable, and primed for swift semantic searches on a variety of data types. However, deploying Weaviate can be a tad complex and time-consuming, especially if you want to hit the ground running.

Enter the aws-cdk-ec2-weaviate project. Using the TypeScript CDK, this project simplifies the process of deploying Weaviate onto an AWS EC2 instance and leverages a Makefile to automate numerous tasks around deployment, configuration, and management.

In this post, we’ll break down the aws-cdk-ec2-weaviate project, diving into its structure and how it employs the AWS CDK, TypeScript, and Makefile to create a streamlined solution for deploying Weaviate on AWS EC2.

AWS CDK #

If you’re not familiar with it, the AWS CDK is an open-source software development framework that facilitates the modeling and provisioning of cloud application resources using familiar programming languages - in this case, TypeScript.

With AWS CDK, you define the necessary infrastructure in TypeScript code inside the lib/aws-cdk-ec2-weaviate-stack.ts file. This infrastructure is then instantiated as an AWS CloudFormation stack when you run the cdk deploy command. The stack comprises the EC2 instance and all the associated resources like the security group, VPC, and IAM roles, necessary for running Weaviate.

If you want to deploy this, start by following the Quickstart in the README.

Project Structure #

The aws-cdk-ec2-weaviate project is structured as follows:

.
├── Makefile                # Task automation
├── README.md
├── bin                     # CDK app entry point
├── cdk.context.json
├── cdk.json
├── config.json             # App configuration
├── docker-compose.yaml     # Weaviate containers
├── lib                     # CDK stack definition
├── package.json
├── (.env)                  # Environment variables    
├── schema.json             # Weaviate schema
├── scripts                 # Utility scripts for Weaviate
├── src                     # EC2 User Data
└── tsconfig.json

Let’s take a closer look at the Makefile and see it’s uses.

The Makefile #

Makefiles can be incredibly useful especially for cloud projects that involve a lot of orchestration of tasks, services and data. In order to use a Makefile you first need to define “targets”. A target can call bash commands, python scripts, or other Makefile targets. Let’s look at the deploy target in the Makefile as an example of this:

deploy: check-env
	$(info [*] Deploying ${APP_NAME} to ${CDK_DEFAULT_ACCOUNT})
	@echo "Creating EC2 key pair..."
	$(MAKE) create-key-pair
	@echo "Deploying CDK stack..."
	@cdk deploy --require-approval never
	$(MAKE) weaviate.wait
	@echo "Creating Weaviate schema..."
	$(MAKE) weaviate.schema.create

The first line deploy: check-env defines the deploy target and it’s dependency check-env. When make deploy is called the script will execute check-env first to ensure that all the environment variables are available. Assuming they are, the script will then execute the rest of the commands in the target.

Makefiles use their own syntax. Bash commands are prefixed with @ and $(MAKE) is used to call other targets. Knowing this we can understand what it is doing in sequence (we will ignore the echo statements since these are just user feedback):

create-key-pair - This target checks for an existing key pair. If none is found it creates it.
cdk deploy --require-approval never - This target deploys the CDK stack. The --require-approval never flag is used to skip the manual approval step.
weaviate.wait - This target waits for Weaviate to be ready. It does this by calling the Weaviate endpoint over HTTP until it returns a 200 status code.
weaviate.schema.create - Once Weaviate is ready this target creates the schema by executing scripts/create_schema.sh.

I would encourage you to look at the logic in the other targets in the Makefile to get a better understanding of how it works. Hopefully this gives you a good idea of how you can use Makefiles to automate tasks in your own projects.

Breaking Down the CDK Code #

Now that we’ve covered the project structure and the Makefile, let’s dive into the CDK code and see how it works. TypeScript CDK projects typically follow a common structure having a bin/ and lib/ directory. The bin/ directory contains the entry point for the CDK app, while the lib/ directory contains the CDK stack definition. Let’s take a closer look at the stack definition in lib/aws-cdk-ec2-weaviate-stack.ts:

import * as config from '../config.json';
import * as cdk from 'aws-cdk-lib';
import { Weaviate } from './vector-database';

export class WeaviateStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const weaviate = new Weaviate(this, 'Weaviate');

  }
}

The line import * as config from '../config.json'; imports the config.json file which contains the configuration for the app. Here is what that file contains:

{
    "layers": {
        "vector_database": {
            "env": {
                "ssh_cidr": "0.0.0.0/0",
                "ssh_key_name": "aws-cdk-ec2-weaviate-key-pair"
            }
        }
    },
    "tags": {
        "org": "my-organization",
        "app": "aws-cdk-ec2-weaviate-key-pair"
    }
}

By using this approach we can make these variables available to the CDK stack at build time. Make sure to avoid using sensitive information in this file as it will be committed to your repository.

The next import lines:

import * as cdk from 'aws-cdk-lib';
import { Weaviate } from './vector-database';

Imports the CDK package and the Weaviate class from the vector-database.ts file. Let’s take a look at that file:

import * as path from 'path';
import * as config from '../config.json';
import cdk = require('aws-cdk-lib');
import { Construct } from 'constructs';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as iam from 'aws-cdk-lib/aws-iam';
import { Asset } from 'aws-cdk-lib/aws-s3-assets';
import * as ssm from 'aws-cdk-lib/aws-ssm';

export class Weaviate extends Construct {
    public readonly vpc: ec2.IVpc;
    public readonly endpointSsmParamName: string;
    public readonly loadAmzOdrTaskSecurityGroup: ec2.ISecurityGroup;
    constructor(scope: Construct, id: string) {
        super(scope, id);

        // // uncomment to deploy a new VPC
        // this.vpc = new ec2.Vpc(this, 'VPC', {
        //     natGateways: 0,
        //     subnetConfiguration: [{
        //         name: 'Vpc',
        //         subnetType: ec2.SubnetType.PUBLIC,
        //         cidrMask: 24
        //     }],
        //     enableDnsHostnames: true,
        //     enableDnsSupport: true
        // });

        // uncomment to use the existing default VPC
        this.vpc = ec2.Vpc.fromLookup(this, 'VPC', {
            isDefault: true,
          });

        // Weaviate instance security group
        const securityGroup = new ec2.SecurityGroup(this, 'WeaviateSecurityGroup', {
            vpc: this.vpc,
            allowAllOutbound: true,
            description: 'Allow SSH (TCP port 22) in',
        });

        // Allow connections from your IP address (set in config.json)
        securityGroup.addIngressRule(
            ec2.Peer.ipv4(config.layers.vector_database.env.ssh_cidr),
            ec2.Port.tcp(8080),
            'Allow Weaviate access');

        securityGroup.addIngressRule(
            ec2.Peer.ipv4(config.layers.vector_database.env.ssh_cidr),
            ec2.Port.tcp(22),
            'Allow SSH');

        // IAM role for the instance allows SSM access
        const role = new iam.Role(this, 'Role', {
            assumedBy: new iam.ServicePrincipal('ec2.amazonaws.com'),
            managedPolicies: [
                iam.ManagedPolicy.fromAwsManagedPolicyName('AmazonSSMManagedInstanceCore'),
            ]
        });
        
        // Ubuntu 20.04 LTS AMI
        const ami = ec2.MachineImage.fromSsmParameter('/aws/service/canonical/ubuntu/eks/20.04/1.21/stable/current/amd64/hvm/ebs-gp2/ami-id');
        
        // create the instance
        const instance = new ec2.Instance(this, 'VectorDatabase', {
            vpc: this.vpc,
            instanceType: ec2.InstanceType.of(ec2.InstanceClass.M6I, ec2.InstanceSize.XLARGE),
            machineImage: ami,
            securityGroup,
            keyName: config.layers.vector_database.env.ssh_key_name,
            role,
            instanceName: 'amz-odr-vector-database',
            blockDevices: [{
                deviceName: '/dev/xvda',
                volume: ec2.BlockDeviceVolume.ebs(16)
            }]
        });

        // add the user data script
        const userData = new Asset(this, 'UserData', {
            path: path.join(__dirname, '../src/config.sh')
        });

        const localPath = instance.userData.addS3DownloadCommand({
            bucket: userData.bucket,
            bucketKey: userData.s3ObjectKey
        });

        instance.userData.addExecuteFileCommand({
            filePath: localPath,
            arguments: '--verbose -y'
        });
        userData.grantRead(instance.role);

        // create an elastic IP and associate it with the instance
        const eip = new ec2.CfnEIP(this, 'EIP', {
            domain: 'vpc'
        });

        // associate the EIP with the instance
        new ec2.CfnEIPAssociation(this, 'EIPAssociation', {
            allocationId: eip.attrAllocationId,
            instanceId: instance.instanceId
        });

        // SSM parameters
        const instanceIdSsmParam = new ssm.StringParameter(this, 'InstanceId', {
            parameterName: `/${config.tags.org}/${config.tags.app}/InstanceId`,
            simpleName: false,
            stringValue: instance.instanceId
        });

        const endpointValue = `http://${eip.attrPublicIp}:8080`
        const endpointSsmParam = new ssm.StringParameter(this, 'WeaviateEndpointParam', {
            parameterName: `/${config.tags.org}/${config.tags.app}/WeaviateEndpoint`,
            simpleName: false,
            stringValue: endpointValue
        });
        this.endpointSsmParamName = endpointSsmParam.parameterName
        new cdk.CfnOutput(this, 'WeaviateEndpointOutput', { value: endpointValue });
    }
}

Make sure to read through it carefully and understand what it does. If you have questions, refer to the CDK TypeScript docs.

EC2 User Data #

User data allows you to configure a new instance by running a custom script on startup. In this case, we use it to install Weaviate and configure it to run as a service. The script is located in src/config.sh and looks like this:

#!/bin/bash -xe

DEBIAN_FRONTEND=noninteractive 
apt-get update -y
# Mounting EBS to m6i
mkfs -t ext4 /dev/nvme1n1
mkdir /data
mount /dev/nvme1n1 /data
cp /etc/fstab /etc/fstab.bak
echo '/dev/nvme1n1 /data ext4 defaults,nofail 0 0' | sudo tee -a /etc/fstab
mount -a
# Install Docker Compose
apt update -y
apt install ca-certificates curl gnupg lsb-release -y
mkdir /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
    $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list
apt-get update -y
apt install docker-ce docker-ce-cli containerd.io docker-compose-plugin -y
usermod -a -G docker ubuntu
# Install Weaviate using Docker Compose
curl -o /home/ubuntu/docker-compose.yaml "https://configuration.weaviate.io/v2/docker-compose/docker-compose.yml?generative_cohere=false&generative_openai=false&generative_palm=false&gpu_support=false&media_type=text&modules=modules&ner_module=false&qna_module=false&ref2vec_centroid=false&runtime=docker-compose&spellcheck_module=false&sum_module=false&text_module=text2vec-transformers&transformers_model=sentence-transformers-multi-qa-MiniLM-L6-cos-v1&weaviate_version=v1.19.8"
sleep 1
# Configure Weaviate to automatically restart and persist data
awk '
  /^  weaviate:$/ {
    print
    print "    restart: always"
    print "    volumes:"
    print "      - /data/weaviate:/var/lib/weaviate"
    while(getline && $0 !~ /^  /);
    if ($0 ~ /^  /) {
      print
    }
    next
  }
  /^  t2v-transformers:$/ {
    print
    print "    restart: always"
    while(getline && $0 !~ /^  /);
    if ($0 ~ /^  /) {
      print
    }
    next
  }
  /CLUSTER_HOSTNAME: '\''node1'\''/ {
    print
    print "      AUTOSCHEMA_ENABLED: '\''false'\''"
    next
  }
  /restart: on-failure:0/ {
    next
  }
  1' /home/ubuntu/docker-compose.yaml > /home/ubuntu/docker-compose-temp.yaml && mv /home/ubuntu/docker-compose-temp.yaml /home/ubuntu/docker-compose.yaml
# Start Weaviate
cd /home/ubuntu && docker compose up -d

Again, make sure to read through it carefully and understand what it does. Feel free to adapt this script to your needs when setting up your own Weaviate configuration. For more information on what is possible I recommend looking at their documentation.

Deployment #

A few prequisites are required before you can deploy the stack:

Install the AWS CLI
Configure the AWS CLI with your credentials and region
Install Node.js and npm
Install TypeScript: npm install -g typescript
Install the AWS CDK Toolkit globally: npm install -g aws-cdk
Install jq using Homebrew or another method of your choice
Run npm install in the project root directory

Once that is out of the way, the project-specific configuration requires you to update .env (you can use the .env.example file as a template) and config.json. Make sure to refer to the README for more details.

Finally, you can deploy the stack using the following command:

make deploy

We saw how this worked earlier. It will take around 10 minutes to complete. Once Weaviate is ready, you can navigate to the endpoint URL using a web browser. For more information on the REST API used by Weaviate, refer to the RESTful API docs. For example, to see the schema, you can navigate to http://<endpoint>/v1/schema and to see the objects, you can navigate to http://<endpoint>/v1/objects.

Conclusion #

aws-cdk-ec2-weaviate gives you a quick way to set up Weaviate on AWS. We’ve examined how to use a Makefile to handle deployment and other tasks as well as dived into the code a little bit.

If you use the project and find it useful, please consider starring it on GitHub. If you have any questions or feedback, feel free to post an Issue, create a Pull Request or reach out to me on Twitter. Thanks for reading and happy building!

References #

A Template for Effortless CDK Pipeline Integration

June 14, 2023·1363 words·7 mins

aws cdk typescript