Python on Blog - uTIcARdI

An incompatible issue of xls file

Mon, 02 Dec 2019 00:00:00 +0000

Background

An integration project: exporting a csv file from system A, convert the csv file to xls file, and import the converted file into system B (A 3rd party system).

Environment:

Python3.7.5
xls library: xlwt1.3
Developing on Mac 10.15.1 with VS Code, and finally the project will be published to a Docker environment(ritht now, it’s Python:3.7.5-alpine3.10).

Issue

The converted xls file, system B doesn’t support it. However, the file can be opened in Microsoft Excel, and then system B supports it after the open operation (did nothing, just open it then close)😫.

VSCode as an IDLE for Django

Fri, 15 Nov 2019 00:00:00 +0000

This blog is about using MS VSCode as Django IDLE (OS: Mac OX, the steps are referenceable on Linux, Windows).

1. Download installation files and install MS VScode, Python from following links:

Optional: On Mac OS, I recommend homebrew for Python Installation because third party libraries’ dependency will be processed smoothly.

Install homebrew:
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Install Python3:
brew install python@3

2. Python Extension

Launch VSCode and install the Extension: Python

os.path.join() or os.sep.join()?

Tue, 23 Jul 2019 00:00:00 +0000

There are two functions join a full path. E.g.: In drive: C:

|- test
| |- test.xt

os.path.join

import os
fullpath = os.path.join('c:', 'test', 'test.txt')

Output on windows:

c:test\test.txt

Output on Mac OS

c:/test/test.txt

os.sep.join

import os
fullpath = os.sep.join(['c:', 'test', 'test.txt'])

Output on windows:

c:\test\test.txt

Output on Mac OS

c:/test/test.txt

From the outputs, you can see the difference: os.path.join() doesn’t work as expected on windows.

Further verification:

with open(fullpath) as test:
 test.read()

Run with error:

 with open(fullpath) as test:
FileNotFoundError: [Errno 2] No such file or directory: 'c:test\\test.txt'

Post a file to Flask by Curl via API

Wed, 20 Feb 2019 00:00:00 +0000

Curl is widely used in command lines or scripts to transfer data.
Different ways to use curl to post a file to a web server with samples (from both client-side and server: Python-Flask side) will be discussed because it took me a while on the server-side to receiving the data from client.
I would like to add samples of Python-requests, Postman later.

Started with

macOS Mojave
Python 3.7.2
Flask 1.0.2
Flask-RESTful 0.3.7
curl 7.54.0 (x86_64-apple-darwin18.0)

The Flask code:

# Py file name: simple.py
from flask import Flask, request
from flask_restful import Api, Resource, reqparse
import werkzeug

app = Flask(__name__)
api = Api(app)

class kzwebfile(Resource):

 def post(self, filename):
pass

api.add_resource(kzwebfile, '/<string:filename>')

if __name__ == "__main__":
 app.run()

Python operator:single star(*)

Sat, 09 Feb 2019 00:00:00 +0000

with Zip

Read a block of pseudo Python code like

comA = zip(lsA, lsB)
comB = zip(*comA)

then made a try to see how * works:

1
2
3
4
5
6
7
8


a = [1, 2, 3]
b = [4, 5, 6]
c = zip(a,b)
for j in c:
print(j)
d = zip(*c)
for i in d:
print(i)

aiohttp/aiodns-Resolving using custom nameservers

Wed, 23 Jan 2019 00:00:00 +0000

aiohttp is an Asynchronous HTTP client/server framework for asyncio and Python

I tried it because of the features:

With this, verifying the CDN’s POPs with different ISPs’ DNS on single a point should be possible.

Since the code on the official document is simple:

from aiohttp.resolver import AsyncResolver

resolver = AsyncResolver(nameservers=[“8.8.8.8”, “8.8.4.4”]) conn = aiohttp.TCPConnector(resolver=resolver)


I tried it with following code:

~~~python
import aiohttp
import asyncio
from aiohttp.resolver import AsyncResolver

resolver = AsyncResolver(nameservers=["114.114.114.114"])

async def dnstesting():
 async with aiohttp.ClientSession(connector=aiohttp.TCPConnector(verify_ssl=False, use_dns_cache=False, resolver=resolver)) as session:
 r = await session.get("http://google.com")
 print(r.text)

asyncio.run(dnstesting())

Run with following error:

get() in get() method of Pyhon dict

Wed, 09 Jan 2019 00:00:00 +0000

Write down the tip for the practice: The Time in Words.

A dict was created for the time in number format to string format:

intToStrDict = {
 1: 'one',
 2: 'two',
 3: 'three',
 4: 'four',
 5: 'five',
 6: 'six',
 7: 'seven',
 8: 'eight',
 9: 'nine',
 10: 'ten',
 11: 'eleven',
 12: 'twelve',
 13: 'thirteen',
 14: 'fourteen',
 15: 'quarter',
 16: 'sixteen',
 17: 'seventeen',
 18: 'eighteen',
 19: 'nineteen',
 20: 'twenty',
 30: 'half',
 40: 'fourty',
 50: 'fifty'
 }

get() is a good way get the value of a key from the dict, so I tried following function to convert minute in number format to string format:

AWS S3 Bucket Creation

Thu, 27 Dec 2018 00:00:00 +0000

Try to create a bucket with the Python library: boto3

from boto3.session import Session
session=Session(aws_access_key_id='key1', aws_secret_access_key='key2')
s3 = session.resource('s3')
s3.create_bucket(Bucket='kz-testbucket')

but failed to run the script with errors:

botocore.exceptions.ClientError: An error occurred (IllegalLocationConstraintException) when calling the CreateBucket operation: The unspecified location constraint is incompatible for the region specific endpoint this request was sent to.
botocore.errorfactory.BucketAlreadyExists: An error occurred (BucketAlreadyExists) when calling the CreateBucket operation: The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again.

Python operator: double start(**)

Fri, 21 Dec 2018 00:00:00 +0000

Here is the data:

testdata = {
 'a' : 1,
 'b': 2
}

There are two ways to pass the data to a function

Passing the data to the function

def loop(data):
 for key, value in data.items():
 print(key, value)

loop(testdata)

Passing the data with operator: **

def loopStar(**kwargs):
 for key, value in kwargs.items():
 print(key, value)

loopStar(**testdata)

Ansible-Access Network via Netconf

Thu, 11 Oct 2018 00:00:00 +0000

Continue to try ansible with the device which was configure in netconf feature on Cisco IOS/IOSXE.

Ansible Inventory:

---
netconf:
hosts:
 netconfIos:
 ansible_host: "172.16.254.100"
 ansible_network_os: "ios"
 ansible_ssh_port: 22
 netconfJunos:
 ansible_host: "172.16.254.9"
 ansible_network_os: "junos"
 ansible_ssh_port: 830
vars:
 ansible_connection: "netconf"
 ansible_ssh_user: test
 ansible_ssh_pass: test123

Note:

ansible_connnection should be ‘netconf’
Be careful on the netconf port, it should be 22 if you enable netconf over SSH (the default port is 830)

PlayBook:

- name: IOS
hosts: netconfIos
gather_facts: false
tasks:
 - name: Get config for IOS devices
 netconf_get:
 source: running
 register: iosnetconf

 - name: Display output
 debug:
 msg: "{{ iosnetconf.stdout }}"

Note:

Netconf feature on Cisco IOS/IOSXE

Sun, 07 Oct 2018 00:00:00 +0000

I tried to see how Ansible works with the netconf feature on Cisco and Juniper in past days: Juniper’s official document is clear and easy to follow. but for Cisco, I followed NETCONF over SSHv2, unfortunately, I was stuck on it for 6 days.

OS Version

Cisco IOS XE Software, Version 16.04.01
Cisco IOS Software, Linux Software (I86BI_LINUXL2-IPBASEK9-M), Experimental Version 15.2(20170809:194209) [dstivers-aug9_]

NetConf Requests

Tried many requests, all failed:

The hello (works) request:

Ansible-Access Network via SSH

Tue, 04 Sep 2018 00:00:00 +0000

After Telnet, continue to discuss how to access network device via SSH with Ansible.

For SSH access, we should consider 3 methods:

Username/password
Username/Public-Key without password
Username/Public-Key with password

Same EVE lab as the Telnet one: The Cisco IOS Router-192.168.1.99 is for SSH

Topo

Enable SSH on the router

Jump #1 username/password part to public key configuration, I discussed how to configure public key on IOS router with Putty before, instead of windows/putty, openssh is useful for generating keys on MAC OS.

Ansible-Access Network via Telnet

Tue, 04 Sep 2018 00:00:00 +0000

I’m learning Ansible by reading the official document and some books like Ansible入门. Most of the instructions show you how to deploy or maintain the services on Linux, or file operations. But what I do is trying to explore its capability on network device management automation.

The most important thing to practice routers/switches/routing protocol is accessing the device and configure it via CLI when I started to learn network, so I start my ansible tour by accessing the network device from CLI too.

Mininet-ImportError: customClass

Fri, 20 Jul 2018 00:00:00 +0000

I tried to run miniedit.py after I have mininet installed, unfortunately, I got the error:

ImportError: cannot import name customClass.

Error

Checked the py file: ~/mininet/mininet/util.py and miniedity.py, find nothing is wrong, search on the Internet, got no clue.

Suddenly I found the version is not the version I wanted to install (I forgot to take a screenshot, the version should be 2.2.2daily something).

Flask, Flask-WTF, Flask-Bootstrap, Select2, jinja2 integration

Mon, 16 Apr 2018 00:00:00 +0000

I intend to acquire the Python skill on web after years’ work on data processing, so I started with Flask by the following Miguel Grinberg’s instruction: The Flask Mega, you can learn how to integrate bootstrap with your project from it.

After that, you can learn how to integrate select2 with your project by reading Flask Web开发:用Select2实现类似知乎的标签系统 as a guide.

Python SMTPlib/Email

Tue, 13 Feb 2018 00:00:00 +0000

I followed the instruction which is written in Chinese to learn how to send emails by Python, here some tips from my practice:

Name of the attachment is shown as ‘ATT00002’

The name of the attachment in the receiver mailbox is shown in the format ‘ATT0000X‘ with the code:
```
att1["Content-Disposition"] = 'attachment; filename="runoob.txt"'
```
Issue

The issue can be fixed by using .addheader() function:

30分钟上手Python网络爬虫

Tue, 30 Jan 2018 00:00:00 +0000

原文链接: https://hackernoon.com/30-minute-python-web-scraper-39d6d038e5da

此翻译已经获得作者许可。对原文、本文任何形式的转载、引用，必须知会作者

我想用Python和Selenium做网络爬虫有一会了，但是一直没有实施。几天之前，我决定小试一下。看起来令人望而却步，但实现代码从Unsplash抓取一些漂亮的图片却是相当容易的。

简易图片爬虫的食材

Python(3.6.3或更新的版本)
Pycharm(社区版就够了)
pip install requests Pillow selenium
geckodriver
Mozlla Firefox
互联网
30分钟(或更少）

*译者注：

OverflowError: signed char is greater than maximum

Tue, 06 Jun 2017 00:00:00 +0000

Here the pseudo code about the flow:

from array import array
c = array('b')
with open(file, mode = 'rb') as f:
 for character in f:
 res = func(character)
 c.append(res)

It runs into the error:

OverflowError: signed char is greater than maximum

with a special character ‘À’ which its binary value is 192:

Character

My way to resolve this error is to update the type code of the array from ‘b’ (signed char) to ‘B’ (unsigned char).

Run Ansible on Windows

Tue, 29 Nov 2016 00:00:00 +0000

Background:

Here is a topology about the environment for performance testing, let’s name the servers 1st:

1 windows server as the License/Workspace server
5 windows servers as the Network Server
5 windows servers as the Automation server

Topo

It took us a lot of time to deploy the product and collect logs on all of these servers.

VSCode for Python debug

Thu, 10 Nov 2016 00:00:00 +0000

MS VS Code for Python debug from k z

View the article here if the iframe failed to load: www.slideshare.net/slideshow/embed_code/key/3GrRaqBAXV9mK

Loop a dict to update the keys

Thu, 21 Jul 2016 00:00:00 +0000

Source dict:

{'keyA': '', 'keyB': ''}

Expected dict:

{‘keyA/'keyA' :’’, keyA/keyB’ : ‘’}

Code1:

keyPrefix = 'keyA'
for key, value in Dict.items():
    newkey = '/'.join([keyPrefix, key])
    Dict[newkey] = Dict.pop(key)

Code2:

keyPrefix = 'keyA'
for key, value in Dict.keys():
    newkey = '/'.join([keyPrefix, key])
    Dict[newkey] = Dict.pop(key)

Result of code1/code2 is:

{'keyA/keyA/keyB' : '', 'keyA/keyA/keyA': ''}

It doesn’t work as expected, and I found the case only exists when prefix is the same to a key of the dict which means code1/code2 works well when the keyPrefix is neither ‘keyA’ nor ‘beyB’.

Python多版本, Sublime Text, Anaconda指北

Tue, 07 Jun 2016 00:00:00 +0000

介绍

Sublime text是一款很好用的文本编辑器，更是一款先进的代码编辑器，支持通过插件进行功能扩展。在使用过UltraEditor, Notepad++, vim等等编辑器后，感觉sublime text最好用 (Update @ 2019-02-07: 目前由Sublimetext转移至MS VSCode)。

SNMP simulator installation guide

Wed, 13 Nov 2013 00:00:00 +0000

About SNMP Simulator

The the SNMP Simulator software is intended for testing SNMP Managers against a large number of SNMP Agents that represent a potentially very large network populated with different kinds of SNMP-capable devices.

Refers to its project page for more info.

Why?

With SNMP simulator you can simulate a device by the output of SNMPwalk without considering MIB files.

This article is about how to install SNMP Simulator.

Install Python