Post a file to Flask by Curl via API

Curl is widely used in command lines or scripts to transfer data.
Different ways to use curl to post a file to a web server with samples (from both client-side and server: Python-Flask side) will be discussed because it took me a while on the server-side to receiving the data from client.
I would like to add samples of Python-requests, Postman later.

• Started with

  • macOS Mojave
  • Python 3.7.2
  • Flask 1.0.2
  • Flask-RESTful 0.3.7
  • curl 7.54.0 (x86_64-apple-darwin18.0)

  The Flask code:

  # Py file name: simple.py
  from flask import Flask, request
  from flask_restful import Api, Resource, reqparse
  import werkzeug
  
  app = Flask(__name__)
  api = Api(app)
  
  class kzwebfile(Resource):
  
      def post(self, filename):
  pass
  
  api.add_resource(kzwebfile, '/<string:filename>')
  
  if __name__ == "__main__":
      app.run()

  Run it:

  (flask) $ python3 simple.py
  * Serving Flask app "simple" (lazy loading)
  * Environment: production
  WARNING: Do not use the development server in a production environment.
  Use a production WSGI server instead.
  * Debug mode: off
  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

• 1st try

  Refer to Tinypng Developer API(I used this method in image optimization for Hugo posts), the file can be uploaded to server in this way:

  curl --data-binary @test.png http://127.0.0.1:5000/test

  Searched on StackOverflow, here is a post about All attributes on the request was summarized by the author of Flask.
  I tried to check the attributions with the following code, but I found request.stream, request.data and request.files affect each other (the content of stream become empty) when I adjusting the order of print syntaxes.

  def post(self, filename):
      print("---stream---\r\n", request.stream.read()) # without read(), it returns <werkzeug.wsgi.LimitedStream object at > all the time
      print("---data---\r\n", request.data)
      print("---args---\r\n", request.args)
      print("---files---\r\n", request.files)
      print("---form---\r\n", request.form)
      # print("---values---\r\n", request.values) # comment this because it just combined args and form.

  Output:

  ---stream---
   b'\x00....<omitted>...'
  ---data---
   b''
  ---args---
  ImmutableMultiDict([])
  ---files---
  ImmutableMultiDict([])
  ---form---
  ImmutableMultiDict([('\x00\x00\x01....<omitted>...', '')])

  Output after print syntaxes were adjusted:

  ---data---
   b''
  ---args---
  ImmutableMultiDict([])
  ---files---
  ImmutableMultiDict([])
  ---form---
  ImmutableMultiDict([('\x00\x00\x01....<omitted>...', '')])
  ---stream---
   b''

  Note: The attribute: request.stream was mentioned in another answer.

• request.form

  Before further steps, I realized request.form receive the data. Checked the request.headers about the content-type:

  def post(self, filename):
      print("---headers---\r\n",request.headers)

  ---headers---
  Host: 127.0.0.1:5000
  User-Agent: curl/7.54.0
  Accept: */*
  Content-Length: 1150
  Content-Type: application/x-www-form-urlencoded
  Expect: 100-continue

  The content type is application/x-www-form-urlencoded which was mentioned in Curl document as well:

  POSTing with curl’s -d option will make it include a default header that looks like Content-Type: application/x-www-form-urlencoded

  Checked request.form: ImmutableMultiDict with request.form.[to_dict()](https://tedboy.github.io/flask/generated/generated/werkzeug.ImmutableMultiDict.html), the key of it is the content of file, the value of it is empty.

  By comparing with coverting the ImmutableMultiDict to list and retrieve the data via index, the option: data-urlencode would be a better choice

  curl --data-urlencode image@test.png http://127.0.0.1:5000/test

  ImmutableMultiDict([('image', '\x00 ...<omitted>...\x00')])

• Checked the POST data again

  Went through Flask official document, and found get_data() is available for incoming data.

  Updated the code to:

  def post(self, filename):
      print("---headers---\r\n", request.headers)
      print("---get_data---\r\n", request.get_data())

  Checked this with content type: Content-Type: application/json:

  curl --data @test.png -H 'Content-Type: application/json' http://127.0.0.1:5000/test

  The data can be received:

  ---get_data---
   b'\x00\x00 ....<omitted>... \x00'

  the Following content types were tested as well:

  • Content-Type: image/png
  • Content-Type: application/octet-stream
  • Content-Type: multipart/form-data
  • Content-Type: text/csv (with a csv file)

  Note: In my testing, request.stream woks as get_data(),differnt to the official document:

  If the incoming form data was not encoded with a known mimetype the data is stored unmodified in this stream for consumption. Most of the time it is a better idea to use data which will give you that data as a string. The stream only returns the data once.

• Option: -F

  Refer the sample, test it with:

  curl -F media=@test.png http://127.0.0.1:5000/test

  the output:

  ---headers---
   Host: 127.0.0.1:5000
  User-Agent: curl/7.54.0
  Accept: */*
  Content-Length: 1351
  Expect: 100-continue
  Content-Type: multipart/form-data; boundary=------------------------80bc4d7df251bbac
  
  ---get_data---
   b'--------------------------80bc4d7df251bbac\r\nContent-Disposition: form-data; name="media"; filename="test.png"\r\nContent-Type: application/octet-stream\r\n\r\n\x00\x00\x01\x00....<omitted>... \x00\r\n--------------------------80bc4d7df251bbac--\r\n'

  files should be the option instead of dealing with the boundary with the get_data():

  def post(self, filename):
      print("---headers---\r\n", request.headers)
      print("---files---\r\n", request.files())

  ---headers---
   Host: 127.0.0.1:5000
  User-Agent: curl/7.54.0
  Accept: */*
  Content-Length: 1351
  Expect: 100-continue
  Content-Type: multipart/form-data; boundary=------------------------c446447e71e96358
  
  ---files---
   ImmutableMultiDict([('media', <FileStorage: 'test.png' ('application/octet-stream')>)])

• Request Parsing

  flask-restful provides a module: Request Parsing to provide simple and uniform access to the variable on flask.request object.

  • command:

    curl --data-urlencode image@test.png http://127.0.0.1:5000/test

    can be supported with:

    parser = reqparse.RequestParser()
    parser.add_argument('image', required=True, help="image filed is required",  location='form')
    
    class kzwebfile(Resource):
    
    def post(self, filename):
        args = parser.parse_args()
        print(args['image'])

  • command:

    curl -F media=@test.png http://127.0.0.1:5000/test

    can be supported with:

    parser = reqparse.RequestParser()
    parser.add_argument('media', type=werkzeug.datastructures.FileStorage, required=True, help="media field is required",  location='files')
    
    class kzwebfile(Resource):
    
        def post(self, filename):
    args = parser.parse_args()
    print(args['media'])

  Check more from this document.

×

Post a file to Flask by Curl via API

$2 $5 Any

Scan the QR code using WeChat

• ← Previous Post
• Next Post →