Turning Musical Instruments into Internet Connected Devices, Part 2

Creating the Socket Server

In the first entry in this series I gave an overview of the different components we’ll be using:

1. MIDI instrument publishing HTTP POST requests through Touch Designer

2. Socket Server connecting Publisher and Subscriber devices

3. Website that subscribes to socket events and renders responsive visuals with OpenGL

In this entry I will be covering the creation of the Socket Server. Even though it’s not the entrypoint of the “stack” so to speak, it forms the backbone of the project and is what allows the different pieces to communicate across the internet.

So let’s get started!

First, you’ll need an AWS account with creds accessible from the terminal. Running…

cat ~/.aws/credentials

Should show a [default] profile along with aws_access_key and aws_secret_access_key

If you need to set up these values check out Configuration and credential file settings in the AWS CLI

The AWS services we’ll be using are:

AppSync

This is a managed service that allows you to connect apps to data and events with secure, serverless, and performant GraphQL and Publish/Subscribe APIs. The fact that it is a managed service means that we don’t need to provision resources in the cloud, which saves on costs and complexity.

Lambda

Lambda is the AWS serverless compute service. Lambdas are chunks of code that are woken up by an event trigger, execute, and go back to sleep. You only pay for the compute time that is used. For our use case whenever a Publish or Subscribe event takes place, a Lambda serverless function will handle the request and respond with a message that fulfills the GraphQL schema.

Let’s start by looking at our GraphQL schema for the project.

directive @aws_subscribe(mutations: [String!]!) on FIELD_DEFINITION

type MidiNote {
  note: Int!
  velocity: Float!
}

type Mutation {
  publishNote(note: Int!, velocity: Float!): MidiNote
}

type Subscription {
  subscribe2Notes: MidiNote @aws_subscribe(mutations: ["publishNote"])
}

type Query {
  subscriberCount: Int!
}

schema {
  query: Query
  mutation: Mutation
  subscription: Subscription
}

First we define the aws_subscribe directive for mutations and declare that subscriptions can be used within field definitions. Look in the Subscription type to see how this directive is being used.

There are then four types in our schema:

1. MidiNote, which contains the note and velocity types from the instrument.

2. Mutation, which is GraphQL-speak for “an operation that transmits information to the API.” Our specific mutation is publishNote. It requires an input of a Note integer and a Velocity floating point decimal. (5, 1.5) would be a legitimate argument set for this mutation. In our application these values will be coming from the MIDI device, which will be covered in a future entry.

3. Subscription, which is GraphQL-speak for “an operation where a listener application tells the API it would like to receive published messages.” This subscription is for the publishNote mutation.

4. Query, a GET operation in GraphQL. This operation is optional for our purposes but can be implemented for operations like retrieving the number of current subscribed devices.

Great! With this GraphQL schema written we can tackle the next piece of the Socket Server: infrastructure.

Instead of clicking through the console and manually selecting services I prefer to use the AWS Cloud Development Kit. This way our entire application can be built or torn down with a single command. Managing cloud resources in this way is helpful when the scope of services becomes large.

import * as appsync from 'aws-cdk-lib/aws-appsync';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
import { type Construct } from 'constructs';
export class SocketVisualsStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const api = new appsync.GraphqlApi(this, 'Api', {
      name: 'MIDISocketServer',
      schema: appsync.SchemaFile.fromAsset('infra/graphql/schema.graphql'),
    });
    
    const subscriptionLambdaRole = new iam.Role(
      this,
      'SubscriptionLambdaExecutionRole',
      {
        assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com')
      }
    );

    const mutationLambdaRole = new iam.Role(
      this,
      'MutationLambdaExecutionRole',
      {
        assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com')
      }
    );

    // Define a Lambda function for handling subscriptions
    const subscriptionLambda = new lambda.Function(this, 'subscriptionLambda', {
      runtime: lambda.Runtime.NODEJS_18_X,
      handler: 'index.handler',
      code: lambda.Code.fromAsset('src/lambdas/subscriptionLambda/dist'),
      role: subscriptionLambdaRole
    });

    // Define a Lambda function for handling mutation
    const mutationLambda = new lambda.Function(this, 'mutationLambda', {
      runtime: lambda.Runtime.NODEJS_18_X,
      handler: 'index.handler',
      code: lambda.Code.fromAsset('src/lambdas/mutationLambda/dist'),
      role: mutationLambdaRole
    });

    // Attach lambdas to api data source
    const mutationDataSource = api.addLambdaDataSource(
      'MutationDataSource',
      mutationLambda
    );

    const subscriptionDataSource = api.addLambdaDataSource(
      'SubscriptionDataSource',
      subscriptionLambda
    );

    subscriptionDataSource.createResolver('subscribe2Notes', {
      typeName: 'Subscription',
      fieldName: 'subscribe2Notes'
    });

    mutationDataSource.createResolver('publishNote', {
      typeName: 'Mutation',
      fieldName: 'publishNote'
    });
  }
}

Let’s go through each part piece by piece.

• SocketVisualsStack is the name of our CDK Stack

  • A Stack is a collection of provisioned resources that are deployed together.

• We then make a new AppSync API with the name MIDISocketServer and include a path to the schema that we created above.

• Next are the two Lambda serverless function definitions. These include:

  • Which language the Lambdas are using (in this case Node v18)

  • The entrypoint to the function source code (index.handler)

  • The location of the source code within the repo

  • Finally, the IAM Role assigned to the Lambda

• mutationDataSource and subscriptionDataSource are defined and attached to our AppSync API via api.addLambdaDataSource

  • You may notice that we did not attach a datasource to our Query definition - this is because we aren’t implementing a query but the GraphQL schema requires one be defined.

• Finally, we can attach Lambda functions as resolvers for each datasource.

The Lambda serverless functions are quite simple.

Mutation lambda handler.ts

export const handler = async (event: any, context: any): Promise<any> => {
  try {
    if (event.info.parentTypeName === 'Mutation') {
      switch (event.info.fieldName) {
        case 'publishNote':
          return {
            note: event.arguments.note,
            velocity: event.arguments.velocity
          };
      }
    }
  } catch (e) {
    console.log(e);
  }
};

Subscription Lambda handler.ts

export const handler = async (event: any, context: any): Promise<any> => {
  try {
    if (event.info.parentTypeName === 'Subscription') {
      switch (event.info.fieldName) {
        case 'subscribe2Notes':
          return {
            note: 0,
            velocity: 0.0
          };
      }
    }
  } catch (e) {
    console.log(e);
  }
};

These functions use a switch statement to execute code based on information in the event. With more complex APIs there could be many cases and types of events here.

For the complete project structure I recommend you clone the parent repository here. I’ve included some deployment scripts that make things easier as well.

After cloning, in the root of the repository run

yarn && yarn deploy

After a successful deployment, go to the AWS Console and navigate to the newly created MIDISocketServer AppSync API. Then, open two windows:

• On the left, run a Subscription

• On the right, publish a Mutation

The MidiEvent values from the publishNote mutation should arrive in the left Subscription window. If you see this result it means the Pub/Sub implementation is working!

For our purposes, this means that anything publishing messages to our AppSync server (right window) can be seen by anything subscribing to that publisher (left window).

In our case, that is going to be a Touch Designer/MIDI instrument publisher and an OpenGL website subscriber. With this technical part out of the way we get to dive into the fun part: making cool things that talk over the internet!

Thanks for reading and stay tuned =]