Get assets page by page in Python

[MartinMeduna]

Recently, someone asked me for a Python example of downloading asset data page by page. This is my very first Python code, so feel free to put comments on what I should do differently or if I break any Python convention 😁

#Set enviroment vailables LS_SITE_ID and LS_IDENTITY_CODE
import http.client
import json
import os

conn = http.client.HTTPSConnection("api.lansweeper.com")

def getAssetData(identity_code, site_id, limit, page, cursor):
    if page=="FIRST":
        assetPagination="limit: %d, page: FIRST" % (limit)
    else:
        assetPagination="limit: %d, page: NEXT, cursor: \"%s\"" % (limit,cursor)

    query="""query getAssetResources {
    site(id: "%s") {
        assetResources(
        assetPagination: { %s }
        fields: [
            "assetBasicInfo.name"
            "assetBasicInfo.userDomain"
            "assetBasicInfo.description"
            "url"
        ]
        ) {
        total
        pagination {
            limit
            current
            next
            page
        }
        items
        }
    }
    }""" % (site_id,assetPagination)

    payload={
        "query": query
    }

    headers = {
    'Content-Type': 'application/json',
    'Authorization': f'Token {identity_code}'
    }
    conn.request("POST", "/api/v2/graphql", json.dumps(payload), headers)
    res = conn.getresponse()
    data = res.read()
    return json.loads(data.decode("utf-8"))


def main():
    allAssets=[]
    site_id=os.environ.get('LS_SITE_ID')
    identity_code=os.environ.get('LS_IDENTITY_CODE')
    print("Loading the first page")
    response=getAssetData(identity_code,site_id,500,"FIRST","")
    allAssets.extend(response['data']['site']['assetResources']['items'])
    while response['data']['site']['assetResources']['pagination']['next']!=None:
        print("Loading page with a cursor: %s " % (response['data']['site']['assetResources']['pagination']['next']))
        response=getAssetData(identity_code,site_id,500,"NEXT" ,response['data']['site']['assetResources']['pagination']['next'])
        allAssets.extend(response['data']['site']['assetResources']['items'])    
    print("Loaded %d assets." % (len(allAssets)))

main()

[BrockKozey]

Your Python code is impressive for a beginner! Consider adding docstrings, error handling, and use consistent string formatting. Structuring your code is a good practice. Well done!

[MartinMeduna]

I will have look but sounds as an excellent advice. Thanks!

[Krish]

Looks great! One thing I would suggest is to use graphQL variables instead of formatting the string, e.g:

Let's say the query starts like this:

query( $cursor: ID, $page: AssetsPage){ 
  site(id: "insert site id here"){
    assetResources(
      assetPagination: {
        limit: 5,
        cursor: $cursor
        page: $page
      },
...

You can put variables at the top by defining them such as $cursor: ID etc. and then you would have a seperate json/dictionary that stores the variable values, e.g:

    variables = {
        "cursor": next_cursor,
        "page": next_page
    }

#next_cursor can be initalized as "" and next_page can be initalized as "FIRST" etc.

And then you can send this as a POST request to LS via whichever you prefer, in my case I use the requests package:

response = requests.request("POST", url, headers=headers, json={"query": query, "variables": variables})
data = response.json()

Make sure to put both the variables and query together as Json format in the post request. This keeps things easier to read and organized, and in cases like yours; you don't need to check if it's FIRST etc. just initialize the cursor as an empty string and then on the next iteration you would put the NEXT page key as the cursor etc.