R scripting: BIM360 metadata to PowerBi via Forge

Question

Currently I have revit files on BIM360 and I am trying to get the metadata into PowerBi. I ran into this code on how to do it but I have some questions that maybe someone can answer them. Currently it appears that the metadata being pulled is from model properties and I need to the metadata from the sheet properties. the code below shows which fields are being pulled:

#######################################################################
## This R script is a sample code that demonstrate how to extract documents' data from a BIM 360 Docs project using Autodesk Forge APIs.
#####################################################################

# Define Forge App Client ID and Secret, BIM 360 Account ID, and Project ID
App_Client_ID <- ""
App_Client_Secret <- ""
BIM360_Account_ID <- ""
BIM360Docs_Project_ID <- ""


#Load libraries required for the R script
library(httr)
library(jsonlite)


#Define a function that loops through BIM 360 Docs folder structure to build a document list 
Parse_Folder <- function(folder_id, folder_name){
  Get_Folder_Content_URL <- paste("https://developer.api.autodesk.com/data/v1/projects/b.",
                                  BIM360Docs_Project_ID,
                                  "/folders/",
                                  folder_id,
                                  "/contents", sep="")
  Get_Folder_Content_Request <- GET(Get_Folder_Content_URL, add_headers("Authorization" = Access_Token))
  Get_Folder_Content_Data <- jsonlite::fromJSON(content(Get_Folder_Content_Request, "text", "application/json", "UTF-8"))
  Folder_Content_Files <- flatten(data.frame(Get_Folder_Content_Data["included"]))
  
  if (nrow(Folder_Content_Files) != 0){
    Folder_Content_Files$document.location <- rep(folder_name, nrow(Folder_Content_Files))
    Folder_Content_Files_SelectedColumns <- as.data.frame(Folder_Content_Files[, c("document.location",
                                                                                   "included.attributes.displayName",
                                                                                   "included.attributes.createUserName",
                                                                                   "included.attributes.createUserId",
                                                                                   "included.attributes.lastModifiedUserName",
                                                                                   "included.attributes.lastModifiedUserId",
                                                                                   "included.attributes.versionNumber",
                                                                                   "included.attributes.fileType",
                                                                                   "included.attributes.extension.type",
                                                                                   "included.attributes.createTime",
                                                                                   "included.attributes.lastModifiedTime",
                                                                                   
    )])
    
    BIM360Docs_Document_List <- rbind(BIM360Docs_Document_List, Folder_Content_Files_SelectedColumns)
    assign("BIM360Docs_Document_List", BIM360Docs_Document_List, envir = .GlobalEnv)
  }
  
  Folder_Content_Data <- flatten(data.frame(Get_Folder_Content_Data["data"]))
  if (nrow(Folder_Content_Data) != 0){
    for(i in 1:nrow(Folder_Content_Data)){
      if(Folder_Content_Data[i,"data.type"]=="folders"){
        tryCatch({
          Parse_Folder(Folder_Content_Data[i,"data.id"], paste(folder_name, "/", Folder_Content_Data[i,"data.attributes.displayName"], sep=""))
        }, error=function(e){})
      }
    }
  }
}


#Use Forge Authentication API to get access token
App_Authenticate <- POST("https://developer.api.autodesk.com/authentication/v1/authenticate",
                         add_headers("Content-Type" = "application/x-www-form-urlencoded"),
                         body=I(list(client_id = App_Client_ID,
                                     client_secret = App_Client_Secret,
                                     grant_type = "client_credentials",
                                     "scope" = "data:read")),
                         encode = "form")
Access_Token <- paste("Bearer", content(App_Authenticate)$access_token,  sep=" ")


#Use Forge Data Managment API to Access Top Folders in the project
Get_Top_Folders_URL <- paste("https://developer.api.autodesk.com/project/v1/hubs/b.",
                             BIM360_Account_ID,
                             "/projects/b.",
                             BIM360Docs_Project_ID,
                             "/topFolders", sep="")
Get_Top_Folders_Request <- GET(Get_Top_Folders_URL, add_headers("Authorization" = Access_Token))
Get_Top_Folders_Data <- jsonlite::fromJSON(content(Get_Top_Folders_Request, "text", "application/json", "UTF-8"))
TopFolders_Content <- flatten(data.frame(Get_Top_Folders_Data))
BIM360Docs_Document_List <- data.frame(Date=as.Date(character()), File=character(), User=character(), stringsAsFactors=FALSE) 

for(i in 1:nrow(TopFolders_Content)){
  TopFolderName <- TopFolders_Content[i,"data.attributes.displayName"]
  if (TopFolderName != "ProjectTb" && TopFolderName != "Photos" && TopFolderName != "Recycle Bin" ){
    tryCatch({
      Parse_Folder(TopFolders_Content[i,"data.id"], TopFolderName)
    }, error=function(e){})
  }
}

names(BIM360Docs_Document_List)[names(BIM360Docs_Document_List)=="document.location"] <- "Document Location"
names(BIM360Docs_Document_List)[names(BIM360Docs_Document_List)=="included.attributes.displayName"] <- "File Name"
names(BIM360Docs_Document_List)[names(BIM360Docs_Document_List)=="included.attributes.createUserName"] <- "Created by"
names(BIM360Docs_Document_List)[names(BIM360Docs_Document_List)=="included.attributes.createUserId"] <- "Created by (ID)"
names(BIM360Docs_Document_List)[names(BIM360Docs_Document_List)=="included.attributes.lastModifiedUserName"] <- "Updated by"
names(BIM360Docs_Document_List)[names(BIM360Docs_Document_List)=="included.attributes.lastModifiedUserId"] <- "Updated by (ID)"
names(BIM360Docs_Document_List)[names(BIM360Docs_Document_List)=="included.attributes.versionNumber"] <- "Version"
names(BIM360Docs_Document_List)[names(BIM360Docs_Document_List)=="included.attributes.fileType"] <- "File Type"
names(BIM360Docs_Document_List)[names(BIM360Docs_Document_List)=="included.attributes.extension.type"] <- "Extension Type"
names(BIM360Docs_Document_List)[names(BIM360Docs_Document_List)=="included.attributes.createTime"] <- "Created Date"
names(BIM360Docs_Document_List)[names(BIM360Docs_Document_List)=="included.attributes.lastModifiedTime"] <- "Updated Date"


# Clear Variables
rm(i, Access_Token,
   App_Client_ID,
   App_Client_Secret,
   App_Authenticate,
   BIM360_Account_ID,
   BIM360Docs_Project_ID,
   Get_Top_Folders_URL, 
   Get_Top_Folders_Request,
   Get_Top_Folders_Data,
   TopFolders_Content,
   TopFolderName,
   Parse_Folder
)

My question is can I use this to get into the Revit file and get the sheet properties? If so how did was the name "versionNumber" in include.attribute.versionNumber decided? Is this how the package in R script predetermines it or is this how the name on the server side appears?

Thank you so much in advance and I'm sorry if I don't have the correct etiquette down packed.

Answer 1

In this R script, it uses the HTTP request endpoint of Forge: GET Folder Contents, which is one call of Data Management API. The link below is the API help about this endpoint. It does not support getting document metadata or properties. https://forge.autodesk.com/en/docs/data/v2/reference/http/projects-project_id-folders-folder_id-contents-GET/

To get metadata or properties, it falls into another category of Forge: Model Derivative API: https://forge.autodesk.com/en/docs/model-derivative/v2/reference/http/urn-metadata-guid-GET/ https://forge.autodesk.com/en/docs/model-derivative/v2/reference/http/urn-metadata-guid-properties-GET/

This blog below may be of a little help for you to understand the endpoints: https://forge.autodesk.com/blog/get-all-dbid-without-enumerating-model-hierarchy

So, that means, you will need to compose the call in R script with the schema of the endpoints above.

However, please be aware:

1. When getting properties, the default limit is 20M. If the response json is larger than 20M, Forge will not return the data by default, you will need to input a query parameter forceget, APII help tells more. So, you may have imagined, the properties can be huge if it is a complicated model. It will take long time when executing R script, or even I am afraid it will be timeout.
2. Actually, when getting folder contents, this sample you are using is only dump the default page. One folder may include a many items. By default, any web service does not return all items in one call, instead, we need to specify how many items that can be exported in one page. in the specific example, the maximum number of one page for item list is 200. In another word, if the folder you check has more than 200, this R script will not tell all documents/sheets.