userver: File uploads and multipart/form-data testing
⚠️ This is the documentation for an old userver version. Click here to switch to the latest version.
All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages Concepts
File uploads and multipart/form-data testing

Before you start

Make sure that you can compile and run core tests as described at Configure, Build and Install.

Take a look at the Writing your first HTTP server and make sure that you do realize the basic concepts.

Step by step guide

It is a common task to upload an image or even work with multiple HTML form parameters at once.

An OpenAPI scheme for such form could look like:

requestBody:
  content: 
    multipart/form-data: # Media type
      schema:            # Request payload
        type: object
        properties:      # Request parts
          address:       # Part1 (object)
            type: object
            properties:
              street:
                type: string
              city:
                type: string
          profileImage:  # Part 2 (an image)
            type: string
            format: binary

Here we have multipart/form-data request with a JSON object and a file.

Let's implement a /v1/multipart handler for that scheme and do something with each of the request parts!

HTTP handler component

Just like in the Writing your first HTTP server the handler itself is a component inherited from server::handlers::HttpHandlerBase:

class Multipart final : public server::handlers::HttpHandlerBase {
public:
// `kName` is used as the component name in static config
static constexpr std::string_view kName = "handler-multipart-sample";
// Component is valid after construction and is able to accept requests
using HttpHandlerBase::HttpHandlerBase;
std::string HandleRequestThrow(
};

The primary functionality of the handler should be located in HandleRequestThrow function. To work with the multipart/form-data parameters use the appropriate server::http::HttpRequest functions:

std::string Multipart::HandleRequestThrow(
const auto content_type =
http::ContentType(req.GetHeader(http::headers::kContentType));
if (content_type != "multipart/form-data") {
req.GetHttpResponse().SetStatus(server::http::HttpStatus::kBadRequest);
return "Expected 'multipart/form-data' content type";
}
const auto& image = req.GetFormDataArg("profileImage");
static constexpr std::string_view kPngMagicBytes = "\x89PNG\r\n\x1a\n";
if (!utils::text::StartsWith(image.value, kPngMagicBytes)) {
req.GetHttpResponse().SetStatus(server::http::HttpStatus::kBadRequest);
return "Expecting PNG image format";
}
const auto& address = req.GetFormDataArg("address");
auto json_addr = formats::json::FromString(address.value);
return fmt::format("city={} image_size={}",
json_addr["city"].As<std::string>(), image.value.size());
}

Note the work with the image in the above snippet. The image has a binary representation that require no additional decoding. The bytes of a received image match the image bytes on hard-drive.

JSON data is received as a string. FromString function converts it to DOM representation.

Warning
Handle* functions are invoked concurrently on the same instance of the handler class. Use synchronization primitives or do not modify shared data in Handle*.

Static config

Now we have to configure the service by providing task_processors and default_task_processor options for the components::ManagerControllerComponent and configuring each component in components section:

components_manager:
    task_processors:                  # Task processor is an executor for coroutine tasks
        main-task-processor:          # Make a task processor for CPU-bound couroutine tasks.
            worker_threads: 4         # Process tasks in 4 threads.
 
        fs-task-processor:            # Make a separate task processor for filesystem bound tasks.
            worker_threads: 1
 
    default_task_processor: main-task-processor  # Task processor in which components start.
 
    components:                       # Configuring components that were registered via component_list
        server:
            listener:                 # configuring the main listening socket...
                port: 8080            # ...to listen on this port and...
                task_processor: main-task-processor    # ...process incoming requests on this task processor.
        logging:
            fs-task-processor: fs-task-processor
            loggers:
                default:
                    file_path: '@stderr'
                    level: debug
                    overflow_behavior: discard  # Drop logs if the system is too busy to write them down.
 
        handler-multipart-sample:
            path: /v1/multipart
            method: POST
            task_processor: main-task-processor

Note that all the components and handlers have their static options additionally described in docs.

int main()

Finally, we add our component to the components::MinimalServerComponentList(), and start the server with static configuration file passed from command line.

int main(int argc, char* argv[]) {
const auto component_list =
return utils::DaemonMain(argc, argv, component_list);
}

Build and Run

To build the sample, execute the following build steps at the userver root directory:

mkdir build_release
cd build_release
cmake -DCMAKE_BUILD_TYPE=Release ..
make userver-samples-multipart_service

The sample could be started by running make start-userver-samples-hello_service. The command would invoke testsuite start target that sets proper paths in the configuration files and starts the service.

To start the service manually run ./samples/multipart_service/userver-samples-multipart_service -c </path/to/static_config.yaml>.

Note
Without file path to static_config.yaml userver-samples-multipart_service will look for a file with name config_dev.yaml
CMake doesn't copy static_config.yaml files from samples directory into build directory.

Now you can send a request to your server from another terminal:

$ curl -v -F address='{"street": "3, Garden St", "city": "Hillsbery, UT"}' \
          -F "profileImage=@../scripts/docs/logo_in_circle.png" \
          http://localhost:8080/v1/multipart
*   Trying 127.0.0.1:8080...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /v1/multipart HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.81.0
> Accept: */*
> Content-Length: 10651
> Content-Type: multipart/form-data; boundary=------------------------048363632fdb9acc
> 
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Tue, 24 Oct 2023 09:31:42 UTC
< Content-Type: application/octet-stream
< Server: userver/2.0 (20231024091132; rv:unknown)
< X-YaRequestId: f7cb383a987248179e5683713b141cea
< X-YaTraceId: 7976203e08074091b20b08738cb7fadc
< X-YaSpanId: 29233f54bc7e5009
< Accept-Encoding: gzip, identity
< Connection: keep-alive
< Content-Length: 76
< 
* Connection #0 to host localhost left intact
city=Hillsbery, UT image_size=10173

Functional testing

Functional tests for the service could be implemented using the service_client fixture from pytest_userver.plugins.core in the following way:

import json
import aiohttp
async def test_ok(service_client, load_binary):
form_data = aiohttp.FormData()
# Uploading file
image = load_binary('logo_in_circle.png')
form_data.add_field('profileImage', image, filename='logo_in_circle.png')
# Adding JSON payload
address = {'street': '3, Garden St', 'city': 'Hillsbery, UT'}
form_data.add_field(
'address', json.dumps(address), content_type='application/json',
)
# Making a request and checking the result
response = await service_client.post('/v1/multipart', data=form_data)
assert response.status == 200
assert response.text == f'city={address["city"]} image_size={len(image)}'

Do not forget to add the plugin in conftest.py:

Full sources

See the full example at: