Appearance
Web Service Specification
Pending Approval
These specifications are pending final approval and may change
IMPORTANT
The Web Service will interact will accept requests from the public internet. Appropriate security measures should be implemented.
The web service is the service which has webhook endpoints which are exposed to the internet. The main purpose of this service is to accept the request and enqueue them for processing. In order to faciliate potentially high volumes of requests, the web service will be built to enable deployment to serverless platforms such as:
Additionally, the application will be containerized so that it can be deployed to a traditional container management platform such as:
Libraries to be utilized
| Library | Use |
|---|---|
| Jak Guru Typescript App Starter | Application Scaffolding and Toolkit |
| @nhtio/logger | Logging |
| @nhtio/kyoo | AMQP Queue |
| luxon | DateTime |
| Koa | Local Web Server |
General Data Flow / Process
The web service will listen for request made to the route:
/:tenant/:vendor/:channel
Once the request is received, it will be added to the incoming queue with a payload similar to:
typescript
import type { KyooSerializable } from '@nhtio/kyoo'
const payload = {
tenant: ":tenant",
vendor: ":vendor",
channel: ":channel",
requestedAt: DateTime.utc()
url: "/:tenant/:vendor/:channel",
headers: {
...
},
payload: KyooSerializable,
trace: {
...
}
}Where the payload has the following attributes:
| Attribute | Type | Description |
|---|---|---|
tenant | string | The identifier of the tenant which the request is being made for |
vendor | string | The vendor of the integration making the request |
channel | string | The communication channel of the client interaction which is being reported. Options are: chat, ticket and voice |
requestedAt | DateTime | The current UTC date/time of the request, using the Luxon DateTime object. |
url | string | The URL of the request. Used to ensure that the request payload is being properly handled. |
headers | object | The key: value representation of the requests's HTTP headers |
payload | string | The requests's body / payload |
trace | object | The Sentry Trace Information |
IMPORTANT
Due to the usage of @nhtio/kyoo, it is not necessary to serialize any part of the object unless it does not match one of the data types found in the @nhtio/serialization documentation. Kyo͞o handles serialization / deserialization automatically. To confirm the serialzability of an object to prevent unexpected errors, you can use the canSerialize utility function exported by Kyo͞o.
The route handler will NOT perform any validation on the request, and will always return a response of 201 with response body of Accepted.
TIP
The reason the route handler will not perform validation on the request is because it will be processed asyncronously by the processor.
Obtaining the Sentry Trace Information
To obtain the information required for the trace, you can use the Sentry.getTraceData(); method. For more information see the following:
Application Configuration
Each instance of the web service will load its requisite configuration through environmental variables which can be set either in .env files or via the mechanisms provided by container orchestration systems.
In addition to any NodeJS-specific environmental variables, the following environmental variables will be used to configure the instance:
| Variable | Type | Description |
|---|---|---|
LOG_LEVEL | enum('emerg', 'alert', 'crit', 'error', 'warning', 'notice', 'info', 'debug') | Determines the minimum level of logs to output to the console |
LOKI_HOST | string | The optional hostname of the Grafana Loki server to push logs to |
LOKI_BASIC_AUTH | string | The optional basic authentication credentials to access Grafana Loki over HTTP |
KYOO_AMQP_PROTOCOL | enum('amqp', 'amqps') | Defines the AMQP protocol to use for the RabbitMQ connection |
KYOO_AMQP_HOSTNAME | string | Defines the AMQP broker host |
KYOO_AMQP_PORT | number | Defines port on which the AMQP broker is listening for requests on |
KYOO_AMQP_USERNAME | string | Defines the username for authenticating with the AMQP broker |
KYOO_AMQP_PASSWORD | string | Defines the password for authenticating with the AMQP broker |
KYOO_AMQP_VHOST | string | Defines the AMQP virtual host |
KYOO_AMQP_EXCHANGE | string | Defines the AMQP message bus exchange |
Runtime Arguments and Flags
Since the Web Service should be designed to work in both serverless and containerized environments, runtime flags will be used to determine if the service should be started as a server.
| Argument / Flag | Alias | Type | Description |
|---|---|---|---|
--serve | -s | boolean | The service should run a local web server to handle requests |
--port | -p | number | Sets the port which the local web server will listen for requests on when the local web server is enabled. Defaults to 3000 if not set. |
shell
node index.mjs -s -p 80Exports from Index
In order to support deployment in a multitude of environments, in addition to the main runtime functionality which will be run by default, the main index will export the following functions:
| Function | Description |
|---|---|
handlerDO | The handler for requests which arrive from a DigitalOcean functions request |
handlerLambda | The handler for request which arrive from an AWS Lambda request |
handlerLocal | The handler for requests which are received by the local webserver |
Functional Considerations
When running in a serverless environment, once the request has been successfully enqueued all connections to the queue broker should be shutdown otherwise the request will needlessly hang. Additionally, it is recommended to also ensure that the logger and Sentry for NodeJS SDK are shut down correctly.
Monitoring and Observability
Logging and observability are used to ensure that the application is operating as expected, and without any major complications or issues. In order to ensure that our monitoring tools and systems are able to inform us when there is an issue, a combination of @nhtio/logger and Sentry for NodeJS will be implemented.
Log Monitoring
In addition to the LOG_LEVEL environmental variable, the logger can be configured to use the Winston Loki Transport by passing at least the LOKI_HOST environmental variable.
Observability
In order to help gain visibility into possible performance issues, unexpected errors or other failures within the application, Sentry for NodeJS will be implemented.
CAUTION
Do not use a Sentry SDK version higher than 7 due to limitations of the NHT Self-Hosted Sentry installation
In order to ensure that sufficient contextual information is available to help pinpoint the origin of an error, Sentry Trace Propagation will be used to connect between a requests's origin within the web service and the attempt of the job which processes that request.
TIP
As this is a single codebase, the Sentry DSN will be hard-coded into the application.
Breakdown of Work
The estimate for the effort needed is 23 hours, broken down as follows:
| Task | Est. Hours | Notes |
|---|---|---|
| Application Scaffolding | 5 | Based on Typescript App Starter |
| Observability Implementation | 2 | @nhtio/logger, Sentry for NodeJS |
| Kyo͞o Integration | 2 | |
| Graceful Shutdown | 2 | Ensures that unless serving, the scripts shutdown cleanly once requests are complete |
| Automated QA Test Creation | 8 | General End to End Functionality Testing |
| Containerization and Packaging | 4 | Dockerization, Building for Serverless, CI/CD Configuration |