Post a file to Flask by Curl via API

Posted by Blog - uTIcARdI on February 20, 2019

Curl is widely used in command lines or scripts to transfer data.
Different ways to use curl to post a file to a web server with samples (from both client side and server: Python-Flask side) will be discussed because it took me a while on server side to receiving the data from client.
I would like to adding samples of Python-requests, Postman later.

  • Started with
    • macOS Mojave
    • Python 3.7.2
    • Flask 1.0.2
    • Flask-RESTful 0.3.7
    • curl 7.54.0 (x86_64-apple-darwin18.0)

    The Flask code:

    # Py file name: simple.py
    from flask import Flask, request
    from flask_restful import Api, Resource, reqparse
    import werkzeug
    
    app = Flask(__name__)
    api = Api(app)
    
    class kzwebfile(Resource):
    
        def post(self, filename):
            pass
    
    api.add_resource(kzwebfile, '/<string:filename>')
    
    if __name__ == "__main__":
        app.run()

    Run it:

    (flask) $ python3 simple.py
    * Serving Flask app "simple" (lazy loading)
    * Environment: production
    WARNING: Do not use the development server in a production environment.
    Use a production WSGI server instead.
    * Debug mode: off
    * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

  • 1st try

    Refer to Tinypng Developer API(I used this method in image optimization for Hugo posts), the file can be uploaded to server in this way:

    curl --data-binary @test.png http://127.0.0.1:5000/test

    Searched on StackOverflow, here is a post about All attributes on the request was summarized by the author of Flask.
    I tried to check the attributions with following code, but I found request.stream, request.data and request.files affect each other (the content of stream become empty) when I adjusting the order of print syntaxes.

        def post(self, filename):
            print("---stream---\r\n", request.stream.read()) # without read(), it returns <werkzeug.wsgi.LimitedStream object at > all the time
            print("---data---\r\n", request.data)
            print("---args---\r\n", request.args)
            print("---files---\r\n", request.files)
            print("---form---\r\n", request.form)
            # print("---values---\r\n", request.values) # comment this because it just combined args and form.

    Output:

    ---stream---
     b'\x00....<omitted>...'
    ---data---
     b''
    ---args---
    ImmutableMultiDict([])
    ---files---
    ImmutableMultiDict([])
    ---form---
    ImmutableMultiDict([('\x00\x00\x01....<omitted>...', '')])

    Output after print syntaxes were adjusted:

    ---data---
     b''
    ---args---
    ImmutableMultiDict([])
    ---files---
    ImmutableMultiDict([])
    ---form---
    ImmutableMultiDict([('\x00\x00\x01....<omitted>...', '')])
    ---stream---
     b''

    Note: The attribute: request.stream was mentioned in another answer.

  • request.form

    Before further steps, I realized request.form receive the data. Checked the request.headers about the content-type:

        def post(self, filename):
            print("---headers---\r\n",request.headers)

    ---headers---
    Host: 127.0.0.1:5000
    User-Agent: curl/7.54.0
    Accept: */*
    Content-Length: 1150
    Content-Type: application/x-www-form-urlencoded
    Expect: 100-continue

    The content type is application/x-www-form-urlencoded which was mentioned in Curl document as well:

    POSTing with curl’s -d option will make it include a default header that looks like Content-Type: application/x-www-form-urlencoded

    Checked request.form: ImmutableMultiDict with request.form.to_dict(), the key of it is the content of file, the value of it is empty.

    By comparing with coverting the ImmutableMultiDict to list and retrieve the data via index, the option: data-urlencode would be a better choice

    curl --data-urlencode image@test.png http://127.0.0.1:5000/test

    ImmutableMultiDict([('image', '\x00 ...<omitted>...\x00')])
  • Checked the POST data again

    Went through Flask official document, and found get_data() is available for incoming data.

    Updated the code to:

        def post(self, filename):
            print("---headers---\r\n", request.headers)
            print("---get_data---\r\n", request.get_data())

    Checked this with content type: Content-Type: application/json:

    curl --data @test.png -H 'Content-Type: application/json' http://127.0.0.1:5000/test

    The data can be received:

    ---get_data---
     b'\x00\x00 ....<omitted>... \x00'

    Following content types were tested as well:

    • Content-Type: image/png
    • Content-Type: application/octet-stream
    • Content-Type: multipart/form-data
    • Content-Type: text/csv (with a csv file)

    Note: In my testing, request.stream woks as get_data(),differnt to the official document:

    If the incoming form data was not encoded with a known mimetype the data is stored unmodified in this stream for consumption. Most of the time it is a better idea to use data which will give you that data as a string. The stream only returns the data once.

  • Option: -F

    Refer the sample, test it with:

    curl -F media=@test.png http://127.0.0.1:5000/test

    the output:

    ---headers---
     Host: 127.0.0.1:5000
    User-Agent: curl/7.54.0
    Accept: */*
    Content-Length: 1351
    Expect: 100-continue
    Content-Type: multipart/form-data; boundary=------------------------80bc4d7df251bbac
    
    ---get_data---
     b'--------------------------80bc4d7df251bbac\r\nContent-Disposition: form-data; name="media"; filename="test.png"\r\nContent-Type: application/octet-stream\r\n\r\n\x00\x00\x01\x00....<omitted>... \x00\r\n--------------------------80bc4d7df251bbac--\r\n'

    files should be the option instead of dealing with the boundary with the get_data():

        def post(self, filename):
            print("---headers---\r\n", request.headers)
            print("---files---\r\n", request.files())
    ---headers---
     Host: 127.0.0.1:5000
    User-Agent: curl/7.54.0
    Accept: */*
    Content-Length: 1351
    Expect: 100-continue
    Content-Type: multipart/form-data; boundary=------------------------c446447e71e96358
    
    ---files---
     ImmutableMultiDict([('media', <FileStorage: 'test.png' ('application/octet-stream')>)])
  • Request Parsing

    flask-restful provides a module: Request Parsing to provide simple and uniform access to the variable on flask.request object.

    • command:

      curl --data-urlencode image@test.png http://127.0.0.1:5000/test

      can be supported with:

      parser = reqparse.RequestParser()
      parser.add_argument('image', required=True, help="image filed is required",  location='form')
      
      class kzwebfile(Resource):
      
          def post(self, filename):
              args = parser.parse_args()
              print(args['image'])

    • command:

      curl -F media=@test.png http://127.0.0.1:5000/test

      can be supported with:

      parser = reqparse.RequestParser()
      parser.add_argument('media', type=werkzeug.datastructures.FileStorage, required=True, help="media field is required",  location='files')
      
      class kzwebfile(Resource):
      
          def post(self, filename):
              args = parser.parse_args()
              print(args['media'])

    Check more from this document.


Post a file to Flask by Curl via API


donation

Scan the QR code using WeChat

comments powered by Disqus