Solved Cloudflare time out error by Transfer Encoding chunked

I had the opportunity to implement an API that needed to handle large data and external integrations within a transaction. It was anticipated that the execution time of the API could take up to about one minute. While the best solution in this case would be to handle it within a batch process, due to various circumstances, it was necessary to address this with a single API.

The system where this was deployed uses Cloudflare, which causes a timeout error if a process exceeds 30 seconds. Although it’s possible to increase the default timeout limit, it’s not a fundamental solution, and increasing the timeout would also prolong the TCP session timeouts, requiring the server to maintain each connection for a longer period. This approach could be disadvantageous from an infrastructure perspective.

Therefore, it was necessary to devise a mechanism specifically for certain APIs to avoid timeout errors. For this purpose, I tried transfer encoding chunked.

Transfer encoding chunked is effective in preventing timeout errors because it allows data to be sent in a series of chunks, rather than waiting for the entire response to be ready before sending. This method keeps the connection active by continuously sending small pieces of data, indicating to the server and any intermediaries (like Cloudflare) that the connection is still alive and data is being transferred.

Sample code

I implemented this feature by Nest.js. Here is the sample code.

  @Post()
  public async sample(@Res() res: Response) {
    res.setHeader('Content-Type', 'application/json');
    res.setHeader('Transfer-Encoding', 'chunked');

    const keepAliveInterval = setInterval(() => {
      console.log('keep alive interval');
      res.write(' '); // Write a space to keep the connection alive
    }, 5000);

    try {
      // Wait for 1 minute (60000 milliseconds)
      // Replace this part with the actual business logic
      await new Promise((resolve) => setTimeout(resolve, 60000));
      res.write(JSON.stringify({ result: 'success!!' }));
    } catch (error) {
      res.write(
        JSON.stringify({
          type: 'SampleApiError',
          errorMessage: 'sample error message',
        }),
      );
    } finally {
      clearInterval(keepAliveInterval);
      res.end();
    }
  }

In this code, every 5 sec, the API returns " “. The response must be string. If you return JSON. You will face the error message below.

TypeError: The "chunk" argument must be of type string or an instance of Buffer or Uint8Array. Received an instance of Object

Pros

As I mentioned, we can deceive timeout limit from services such as Cloudflare.

Cons

Once the server sends the initial headers with a status code (such as 200 OK), it cannot later change that status code if an error occurs during the processing of the remaining data. This means that if an error occurs after the initial headers are sent, the server cannot send a 500 Internal Server Error or a 400 Bad Request status code, as the status code is already fixed.

This behavior necessitates careful handling of errors when using chunked transfer encoding. It’s important to ensure that as much error checking as possible is done before sending the initial headers.

In my use case. all validations and auth checked are run before transfer encoding. Additionally, in the case of errors, I included errorMessage and errorType in the response.

Summary

The use of transfer encoding chunked is a strategic solution to handle long-running processes API in environments with strict timeout constraints, but it requires careful handling of response status and error messages.