Getting started with VbaGit - Desktop Liberation

For an overview of what VbaGit is, please read Integrate VBA with Github.

Page Content hide

1 Getting the code

2 Access to the IDE

3 Getting authenticated on Github

4 Storing credentials in the registry

5 The scenario

6 Your settings

7 Reading from Github

8 Writing to the staging area

9 Committing to Git

10 Issues

11 Technical writeup

Getting the code

You can get the code from github or you can get a premade bootstrap workbook from the github stuff section in the downloads page. This is the code needed to be able to either import code from github into workbooks or to make repos from workbooks. It may be that some less sophisticated antivirus software will dislike downloading the workbook, because it needs to access your code module. If this is the case then you’ll need to build a bootstrap workbook by including all the code from the repo.

Access to the IDE

The whole point of VbaGit is to automatically read and write to the VBA code. There are various security things built in to prevent that – so we have to open those up to allow VbaGit to do its work. I recommend you turn them off again once you have finished with VbaGit. You do that in Options/Trust Center/Macros, and set it like this.

Getting authenticated on Github

VbaGit uses the GITHUB API directly from VBA to commit to and read from GitHub. If you are just reading (importing public github code into Excel), you don’t need to authenticate but if you are planning to write to github, then you need to set up a git hub application. I’m assuming you already have a login to be reading this in the first place. Go to github/settings/applications and create an application. You’ll see a dialog like this which you should complete. You’ll also get a client id, and a client secret. You’ll need those, so make a note of them.

Storing credentials in the registry

I never like to leave credentials in a procedure, so we’ll do a one off store of credentials to your local registry, then forget all about this. You’ll find a proc in the VbaGit that looks like this.

Private Function deleteThisAfterRunningOnce()
' substitute your git application clientid/secret
  setGitBasicCredentials "git user name", "git password"
  setGitClientCredentials "short git creds(id)", "longer git creds(secret)"
End Function

Private Function deleteThisAfterRunningOnce()

' substitute your git application clientid/secret

setGitBasicCredentials "git user name", "git password"

setGitClientCredentials "short git creds(id)", "longer git creds(secret)"

End Function

Substitute in your github login, and the application credentials, run the function, and delete it (or at lease obscure your credentials). Normally you wouldn’t need your github login details for oauth2 authentication, but the api does not allow full oauth2 from non-web applications, so we have to do basic authentication (hence the need for the credentials) to be able to obtain a token to use the API. All this happens behind the scenes anyway. Obviously if you change your login details, you’ll need to update and run this again.

You don’t need to do any of this if you are only reading from GitHub – just if you are planning to commit projects directly from VBA.

The scenario

Regardless of whether you are reading from or writing to your workbook, VbaGit is expecting to be run from a Boostrap workbook, with the other workbook you want to read from or write to open also. So you should have 2 workbooks open. VbaGit will detect which is which.

Your settings

Every user has a different local environment, so before you can start you need to describe yours. You’ll find this function in the VbaGit module. You’ll need to change a few of these to your environment. The most likely changes needed are highlighted in the code.

Public Function getVGSettings(Optional force As Boolean)
    
    ' get the settings - only bothers with the parse once
    If force Or isUndefined(VGSettings) Then
        If (isSomething(VGSettings)) Then
            VGSettings.tearDown
        End If
        Set VGSettings = New cJobject
        With VGSettings.init(Nothing)
            With .add("EXTRACT")
                .add "TO", "c:/users/fhk647/documents/gas/Extraction/Scripts/"  ''' this is the folder that will be your root local repository
            End With
            With .add("GIT")
                With .add("COMMITTER")
                    .add "name", "Bruce McPherson"                              ''' for commit messages on git
                    .add "email", "bruce@mcpher.com"                            ''' for commit messages on git
                End With
                .add "USERAGENT", "brucemcpherson"                              ''' the repo owner
                .add "SCOPES", "repo,gist"
                .add "OWNER", .toString("USERAGENT")
            End With
            With .add("REGISTRY")
                .add "root", "xLiberation"
                .add "app", "vbagit"
                .add "basic", "basichash"
                .add "client", "clienthash"
            End With
            With .add("APP")
                .add "VERSION", "0.0.1"
            End With
            With .add("FILES")
                .add "README", "README.md"
                .add "INFO", "info.json"
                .add "DEPENDENCIES", "dependencies.md"
                .add "CROSS", "cross.md"
            End With
            With .add("FOLDERS")
                .add "SCRIPTS", "scripts"
                .add "DEPENDENCIES", "libraries"
            End With
            With .add("PROJECT")
                .add "NAME", "VbaGitAddOn"
            End With
            With .add("VBA")
                With .add("TYPES")
                    .add "StdModule", 1
                    .add "ClassModule", 2
                End With
            End With
        End With
    End If
    Set getVGSettings = VGSettings
End Function

Public Function getVGSettings(Optional force As Boolean)

' get the settings - only bothers with the parse once

If force Or isUndefined(VGSettings) Then

If (isSomething(VGSettings)) Then

VGSettings.tearDown

End If

Set VGSettings = New cJobject

With VGSettings.init(Nothing)

With .add("EXTRACT")

.add "TO", "c:/users/fhk647/documents/gas/Extraction/Scripts/" ''' this is the folder that will be your root local repository

End With

With .add("GIT")

With .add("COMMITTER")

.add "name", "Bruce McPherson" ''' for commit messages on git

.add "email", "bruce@mcpher.com" ''' for commit messages on git

End With

.add "USERAGENT", "brucemcpherson" ''' the repo owner

.add "SCOPES", "repo,gist"

.add "OWNER", .toString("USERAGENT")

End With

With .add("REGISTRY")

.add "root", "xLiberation"

.add "app", "vbagit"

.add "basic", "basichash"

.add "client", "clienthash"

End With

With .add("APP")

.add "VERSION", "0.0.1"

End With

With .add("FILES")

.add "README", "README.md"

.add "INFO", "info.json"

.add "DEPENDENCIES", "dependencies.md"

.add "CROSS", "cross.md"

End With

With .add("FOLDERS")

.add "SCRIPTS", "scripts"

.add "DEPENDENCIES", "libraries"

End With

With .add("PROJECT")

.add "NAME", "VbaGitAddOn"

End With

With .add("VBA")

With .add("TYPES")

.add "StdModule", 1

.add "ClassModule", 2

End With

End If

Set getVGSettings = VGSettings

End Function

Reading from Github

Once you have the receiving worksheet open, you can run this , substituting the reponame for the repo you are importing from

doImportFromGit "the repo name"

1	doImportFromGit "the repo name"

Writing to the staging area

These updates the staging are you defined in the settings earlier with all the stuff that will be later committed to Git. Note that you only need to specify the main module. All the dependencies will be figured out automatically.

doExtraction "the repoName", "the list of main modules separated by commas"

1	doExtraction "the repoName", "the list of main modules separated by commas"

As an example, I have a large workbook – cDataSet.xlsm – which contains examples about everything in this site. In the past you would have needed to download this (you still can, see Downloads), but if you wanted a specific set of examples or class only, you would have had all the other stuff too. Using VbaGit, I’ve split up as below so that can VbaGit be used to create fresh, empty workbooks with just the code you need.

Public Sub doEverything()

    ' these are the projects in this workbook i want to separate

    ' base classes
    doExtraction "cJobject", "cJobject"
    doExtraction "cDataSet", "cDataSet"
    
    ' utilities
    doExtraction "excelClassSerializer", "classSerializer"
    
    ' example projects
    doExtraction "excelRestLibraryExamples", "restLibraryExamples"
    doExtraction "excelRoadmapper", "doRoadmapper"
    doExtraction "excelGoogleSheets", "googleSheets,googleWireExample"
    doExtraction "excelColor", "heatmapExamples,colorizing"
    doExtraction "excelD3", "D3"
    doExtraction "excelOauth2", "oAuthExamples"
    doExtraction "excelParseCom", "parseCom"
    doExtraction "excelProgressPar", "TestProgressBar"
    
    ' now write them to git
    doGit
    
End Sub

Public Sub doEverything()

' these are the projects in this workbook i want to separate

' base classes

doExtraction "cJobject", "cJobject"

doExtraction "cDataSet", "cDataSet"

' utilities

doExtraction "excelClassSerializer", "classSerializer"

' example projects

doExtraction "excelRestLibraryExamples", "restLibraryExamples"

doExtraction "excelRoadmapper", "doRoadmapper"

doExtraction "excelGoogleSheets", "googleSheets,googleWireExample"

doExtraction "excelColor", "heatmapExamples,colorizing"

doExtraction "excelD3", "D3"

doExtraction "excelOauth2", "oAuthExamples"

doExtraction "excelParseCom", "parseCom"

doExtraction "excelProgressPar", "TestProgressBar"

' now write them to git

doGit

End Sub

Committing to Git

You can either use a standard git client to commit the staging area to gitHub, or run this from within VBA

doGit "the repoName or leave blank to commit everything in the staging area to github"

1	doGit "the repoName or leave blank to commit everything in the staging area to github"

Issues

Occassionally you’ll get an error 409 from the GitHub API when using doGit. This means there is some kind of timing problem going on in the GIT database, as described here – https://developer.github.com/v3/git/ . As far as I can tell the commit actually happens successfully, so it’s likely that I can ignore this error, but for now I will continue to report it as an error. You can run it again if you want to be sure.
If one if your procedures happens to have an argument that shares a name with a public procedure, then that procedure will wrongly be considered to be a dependency and it’s code (and all its own dependencies) will be included in the repo. This is not a massive problem – it just means you’ll get more code than you need. I’ll apply a fix for this whenever I can figure something out.

Now take a look below for more on this subject

Technical writeup

You can find the repository for VbaGit here. I’ll be adding to the technical writeup over time.

Hover over the image below to see the full code for the main VbaGit module is below, and the documentation is here.

Expand to see the code

Option Explicit
' this is based on the ideas from https://ramblings.mcpher.com/drive-sdk-and-github/getting-your-apps-scripts-to-github
' and is about getting your excel code to github
' VbaGit v0.2.3

' settings are in public var
Dim VGSettings As cJobject
'/**
' * example of exporting/importing a repos from github
' */
Public Sub doEverything()
    doExtraction "VbaGit", "VbaGit"
    doGit "VbaGit"
    ' these are the projects in this workbook i want to separate
   'doExtraction "cJobject", "cJobject"
   'doGit "cJobject"

   'doExtraction "vanillacJobject", "advancedcJobject"
    'doExtraction "emptycDataSet", "googleSheets,googleWireExample,oauthexamples,restLibrary"
    'doGit "emptycDataSet"
'
'    ' utilities
'    doExtraction "excelClassSerializer", "classSerializer"
'
'    ' example projects
    'doExtraction "excelRestLibraryExamples", "restLibraryExamples"
'     doExtraction "excelRoadmapper", "doRoadmapper"
'    doExtraction "excelGoogleSheets", "googleSheets,googleWireExample"
'    doExtraction "excelColor", "heatmapExamples,colorizing"
'    doExtraction "excelD3", "D3"
'    doExtraction "excelOauth2", "oAuthExamples"
'    doExtraction "excelParseCom", "parseCom"
'    doExtraction "excelProgressBar", "TestProgressBar"

 '   doExtraction "cChromeTraceVBA", "cChromeTraceVBA,testChromeTrace"
    'doExtraction "cVBAProject", "cVBAProject,cVBAProcedure,cVBAmodule,cVBAArgument"
    'doExtraction "cDataSet", "cDataSet"
    'doExtraction "excelRestLibrary", "restLibrary,cRest"
    ' now write them to git
    'doGit "emptycDataSet"
    'doExtraction "effex-demo-markers-excel", "effexTests,VBAMapsEffex"
    'doGit "effex-demo-markers-excel"
    
End Sub
'/**
' * example of importing a repo from github and replaces the code in the companion wokbook
' */
Public Sub doTheImport()
    ' this is the something I want to import into the companion workbook
    '//doImportFromGit "cDataSet"
    ''doImportFromGit "cJobject"
    
End Sub

' NOTES ON IMPORTING CODE FROM GITHUB
'
' since this is for public code,
' there is no need to register or create a github app (unlike for writing code)
' although if you do have credentials you can apply them if you want
'
' 1 .open the vbagit worksheet and the sheet to which you want to import the code
' 2 .set up do the import
'       first argument is the repoName to get the code from
'       second is the project name - you should probably leave that blank
'       third is whether to apply the excel references - the defaut is not to
' 3. run it and compile the result
' ----------------------------------------
' NOTES ON COMMITTING CODE TO GITHUB

' open the vbagit workbook and set up for your environment if you havent already done it
'
' 1. set your credentials in deleteThisAfterRunningOnce,
'       then either delete or obscure your credentials, you wont need to run this again
'       you need to have set up an app in github and got the api credentials

' 2. set getVGSettings for your environment
'       this describes who you are and where to do things
'       the main ones are
'           EXTRACT.TO (a staging area folder for all your files to be dumped)
'           GIT.COMMITTER
'           GIT.USERAGENT
'       you probably don't need to change any others unless you want to rename or reorganize the repo contents

' vbagit runs as a companion workbook
' you open vbagit
' you open the workbook containing the modules you want to commit
' switch to the vbagit workbook and set up what you want to do
'
' 3. run doextraction for each repo you want to create, naming the module(s) for the repo
'       you can specify a project name if you have more than 2 workbooks open (vbagit + the one containing the modules to be processed)
'       however most people leave their projects with the default name of VBAProject.
'       It will take the first one it finds with the given name - so its best just to have the right workbook open
'       the first arg wil be the reponame - this what it will be called on GIThub
'       the second is the list of modules that are to be part of this repo
'       you just need to name the main module(s). These will end up in the scripts folder
'       Any dependent modules/classes will be automatically detected and added to the libraries folder
'       Module Documentation is also automatically generated at this stage
'       Note that you can create multiple repos from a single workbook
'       see doEverything for an example
'       any shared libraries needed will be detected and committed to whichever repo as required

' 4. when ready you can run doGit to commit everything in the EXTRACT.TO folder up to GIT
'       actually, its the contents of info.json that decides what to copy up. anyting in EXTRACT.TO not created by
'       vbagit will be ignored
'       all repos will be commited at once, except if you specify a repoName to doGit. then it will only do one
'       your readme will only be committed if there is not already one in the Github repo
'       if you prefer to use the git client instead, you can make your EXTRACT.TO location your local git repo

'/**
' * sets up your credentials in the windows registry.
' * should be deleted or obfuscated after running once
' */
Private Function deleteThisAfterRunningOnce()
    ' substitute your git application clientid/secret
    setGitBasicCredentials "username git", "passwrod git"
    setGitClientCredentials "short", "long"
End Function
'/**
' * sets up the settings object if its not already set up and returns it
' * @param {boolean} force whether to force a new set up
' * @return {cJobject} the settings
' */
Public Function getVGSettings(Optional force As Boolean)
    
    ' get the settings - only bothers with the parse once
    If force Or isUndefined(VGSettings) Then
        If (isSomething(VGSettings)) Then
            VGSettings.tearDown
        End If
        Set VGSettings = New cJobject
        With VGSettings.init(Nothing)
            With .add("EXTRACT")
                .add "TO", "c:/users/bruce/documents/gas/Extraction/Scripts/"
            End With
            With .add("GIT")
                With .add("COMMITTER")
                    .add "name", "Bruce McPherson"
                    .add "email", "bruce@mcpher.com"
                End With
                .add "USERAGENT", "brucemcpherson"
                .add "SCOPES", "repo,gist"
                .add "OWNER", .toString("USERAGENT")
            End With
            With .add("REGISTRY")
                .add "root", "xLiberation"
                .add "app", "vbagit"
                .add "basic", "basichash"
                .add "client", "clienthash"
            End With
            With .add("APP")
                .add "VERSION", "0.2.4"
            End With
            With .add("FILES")
                .add "README", "README.md"
                .add "INFO", "info.json"
                .add "DEPENDENCIES", "dependencies.md"
                .add "CROSS", "cross.md"
            End With
            With .add("FOLDERS")
                .add "SCRIPTS", "scripts"
                .add "DEPENDENCIES", "libraries"
            End With
            With .add("PROJECT")
                .add "NAME", "VbaGitAddOn"
            End With
            With .add("VBA")
                With .add("TYPES")
                    .add "StdModule", 1
                    .add "ClassModule", 2
                End With
            End With
        End With
    End If
    Set getVGSettings = VGSettings
End Function
'/**
' * do the import from github and replace the modules in the companion workbook
' * @param {} repoName the github reponame
' * @param {} projectName the vbaproject name
' * @param {} applyExcelReferences whether to apply the excel references in dependency list
' */
Public Sub doImportFromGit(repoName As String, _
    Optional projectName As String = vbNullString, _
    Optional applyExcelReferences As Boolean = False)
    
    ' get all the projects in this workbook
    Dim projects As cJobject, settings As cJobject, project As cJobject
    Dim repo As cJobject, git As cVbaGit, result As cJobject, _
        job As cJobject, info As cJobject
        
    Set settings = getVGSettings(True)
    Set projects = getVbaAsJobject(projectName)
    Set project = projects.children(1)
    
    ' create dependency list - only do the first project for this scope
    If (project.getObject("project").name = settings.toString("PROJECT.NAME")) Then
        MsgBox "you need to open both vbagit and the workbook to set up: you cannot overwrite vbagit"
        Exit Sub
    End If
    
    ' check we are doing the right thing
    If MsgBox("You are 100% sure that you want me to import the code from repo " & repoName & _
        " from github into " & project.getObject("project").wBook.name, vbYesNo) <> vbYes Then
        Exit Sub
    Else
        Debug.Print "Importing the code from repo " & repoName & _
        " from github into " & project.getObject("project").wBook.name
        
        ' get a handle for git api
        Set git = New cVbaGit
        
        ' actually we are using basic authentication
        ' this optional, if you have already setup a github app
        ' it will also get you more quota
        ' if not then you can comment this out
        git.setAccessToken getGitBasicCredentials(), getGitClientCredentials()
        If (Not git.isAccessToken) Then
            Debug.Print "you are using an unauthenticated git connection with limited quota"
        End If
        
        ' get the repo
        Set repo = getRepo(git, repoName).getObject("data")
        
        ' get the info file
        Set result = git.getFileByPath(settings.toString("FILES.INFO"), repo)
        
        ' process the scripts
        Set info = JSONParse(result.toString("content"))
        
        ' the modules
        getCodeFromGit project, git, _
            settings.toString("FOLDERS.SCRIPTS"), info, _
            "modules", repo
        
        ' the libraries
        getCodeFromGit project, git, _
            settings.toString("FOLDERS.DEPENDENCIES"), info, _
            "dependencies", repo
        
        If (applyExcelReferences) Then
            Debug.Print "Apply excel references"
            registerExcelReferences project, info.child("excelReferences")
        End If
        
        info.tearDown
        
    End If

End Sub
'/**
' * get the code from git for a particular module
' * @param {} project the project object
' * @param {} git a handle to the cVbaGit object
' * @param {} folder the folder to find the file in
' * @param {} childName the branch of the project to work from (scripts/libraries)
' * @param {} repo the repo object containing this file
' */
Private Sub getCodeFromGit(project As cJobject, git As cVbaGit, _
        folder As String, info As cJobject, _
        childName As String, repo As cJobject)
    
    Dim job As cJobject, result As cJobject
    Debug.Print "Importing project " & childName
    For Each job In info.kids(childName)
        Set result = git.getFileByPath(folder & "/" & job.toString("fileName"), repo)
        Debug.Assert result.cValue("success")
        replaceModule project, job, result.toString("content")
        result.tearDown
    Next job

End Sub
'/**
' * get the code from git for a particular module
' * @param {} project the project object
' * @param {} infoItem the object from info.json for this file
' * @param {} code the new code to use
' * @return {} whether it was successful
' */
Private Function replaceModule(project As cJobject, infoItem As cJobject, code As String) As Boolean

    Dim jm As cJobject, module As VBComponent, m As cJobject, _
        t As Long, settings As cJobject, vm As cVBAmodule
        
    Set settings = getVGSettings
    
    Set m = project.child("modules").findInArray("name", infoItem.toString("name"))
    
    If (m Is Nothing) Then
        ' we need to create a new module
        Debug.Print "creating "; infoItem.toString("type"); " "; infoItem.toString("name")
        Set module = project.getObject("project") _
            .theProject _
            .VBComponents _
            .add(settings.cValue("VBA.TYPES." & infoItem.toString("type")))
        module.name = infoItem.toString("name")
    Else
        Set vm = m.parent.getObject("module")
        Set module = vm.vCom
        If (vm.textKind <> infoItem.toString("type")) Then
            Debug.Print "module " & vm.name & " is not the same type as on github: cannot replace it"
            Exit Function
        End If
        ' delete existing content
        Debug.Print "replacing "; infoItem.toString("type"); " "; infoItem.toString("name")
        
    End If
    
    'clear current contents
    If module.codeModule.CountOfLines > 0 Then
        ' remove Option Explict lines if it was added automatically
        module.codeModule.DeleteLines 1, module.codeModule.CountOfLines
    End If
    
    ' add the new code
    module.codeModule.AddFromString code
    
    replaceModule = True
End Function
'/**
' * get the code from git for a particular module
' * @param {} git a cVbaGit handle
' * @param {} repoName the name of the repo
' * @param {} complain whether to complain on failure
' * @return {} the repo object
' */
Private Function getRepo(git As cVbaGit, repoName As String, Optional complain As Boolean = True) As cJobject
    Dim settings As cJobject, result As cJobject
    Set settings = getVGSettings()
    
    Set result = git.getSpecificRepo(settings.toString("GIT.OWNER"), repoName)
    If (complain And Not result.cValue("success")) Then
        Err.Raise vbObjectError + result.cValue("code"), result.stringify
    End If
    Set getRepo = result
End Function
'/**
' * extract the files for a particular project and write them to the staging area
' * @param {} repoName the name of the repo
' * @param {} optListOfModules list of main modules to use as starting point
' * @param {} projectName the name of the vba project
' */
Private Sub doExtraction(repoName As String, _
    Optional optListOfModules As String = vbNullString, _
    Optional projectName As String = vbNullString)

    ' get all the projects in this workbook
    Dim projects As cJobject, dependencyList As cJobject, _
        infoJob As cJobject, settings As cJobject, project As cJobject
        
    Set settings = getVGSettings(True)
    Set projects = getVbaAsJobject(projectName)
    
    ' create dependency list - only do the first project for this scope
    If (projects.hasChildren) Then
        Set project = projects.children(1)
        Set dependencyList = getDependencyList(project, repoName, optListOfModules)
    Else
        ' no projects
        MsgBox "no projects detected for repo " & repoName
        Exit Sub
    End If
    
    Debug.Print "extracting repo "; repoName; " from project "; project.getObject("project").theProject.name
      
    ' create the info & cross references file
    Set infoJob = makeInfoFile(project, dependencyList)
    
    ' now we write all the scripts to some staging area
    writeToStagingArea infoJob, dependencyList
    
    ' mark as extracted
    infoJob.child("extracted").setValue True
    writeInfoFile project, infoJob, makeCrossReferenceJob(dependencyList), dependencyList


    ' clean up
    projects.tearDown
    dependencyList.tearDown
    infoJob.tearDown
    settings.tearDown
    Debug.Print "done extracting "; repoName
End Sub
Private Sub testmodulestuff()
    Dim job As cJobject, settings As cJobject, projects As cJobject, proc As cVBAProcedure
    Dim r As Range, rx As RegExp
    Set r = Range("sheet1!a1")
    Set settings = getVGSettings(True)
    Set projects = getVbaAsJobject()
    
    For Each job In projects.children(1).kids("modules")
        With job.getObject("module")
            For Each proc In .procedures
                With proc
                    Set r = r.Offset(1)
                    r.Offset(, 0).value = .name
                    r.Offset(, 1).value = .startLine
                    r.Offset(, 2).value = .lineCount
                    r.Offset(, 3).value = .getTheEndRx.Test( _
                        .codeModule.Lines(.startLine, _
                            .getFinishWithoutTrailingComments - .startLine + 1))
                    r.Offset(, 4).value = .getTheCode
                    r.Offset(, 5).value = .getTheCodePlusLeadingComments
                End With
            Next proc
        End With
    Next job
    projects.tearDown
End Sub
' these are all about committing to Git
'--------------------------------------------------------
'
'/**
' * call this to commit all extracted projects to github them from the staging area
' * @param {} specificRepoName the name of the repo - if blank it will do them all
' */
Private Sub doGit(Optional specificRepoName As String = vbNullString)

    Dim allInfoFiles As cJobject, settings As cJobject, git As cVbaGit, _
        repos As cJobject, result As cJobject
    
    
    Set settings = getVGSettings(True)
    
    ' the info files drive what needs to be written to Git
    Set allInfoFiles = getAllInfoFiles(specificRepoName)
    
    ' get a handle for git api
    Set git = New cVbaGit
    
    ' actually we are using basic authentication
    git.setAccessToken getGitBasicCredentials(), getGitClientCredentials()
    If (Not git.isAccessToken) Then
        MsgBox ("you cannot commit to git without authentication- please set up")
    Else
    ' get all the repos, creating any missing ones, and adding all the files
        Set repos = createRepos(git, allInfoFiles)
    End If
    ' clean up
    allInfoFiles.tearDown
    repos.tearDown
    git.tearDown
    
End Sub
'/**
' * get all known repos belonging to the git logged in individual
' * @param {} git a handle to the cVbaGit api
' * @return {} all the known repos
' */
Private Function getAllTheRepos(git As cVbaGit) As cJobject
    Dim result As cJobject
    Set result = git.getMyRepos
    If (result.cValue("success")) Then
        Set getAllTheRepos = result.getObject("data")
    Else
        MsgBox "failed to get all the repos " + result.stringify
        Exit Function
    End If
End Function
'/**
' * create any repos in our list of info objects that don't exist
' * @param {} git a handle to the cVbaGit api
' * @param {} infos a list of info objects
' * @return {} all the known repos updated
' */
Private Function createRepos(git As cVbaGit, infos As cJobject) As cJobject

    Dim repos As cJobject, info As cJobject, repo As cJobject, result As cJobject, _
        added As Long, settings As cJobject, job As cJobject
    Set settings = getVGSettings
    ' all my repos
    Set repos = getAllTheRepos(git)
    added = 0
    
    ' find any missing
    For Each info In infos.children
        ' we'll only do uncommitted or modified since last commit
        If (info.cValue("committedDate") < info.cValue("modifieddate")) Then
            If (isSomething(repos)) Then
                Set repo = repos.findInArray("name", info.toString("repo"))
            Else
                Set repo = Nothing
            End If
            ' need to create it
            If (repo Is Nothing) Then
                Set result = git.createRepo(info.toString("repo"))
                If (Not result.cValue("success")) Then
                    MsgBox "error creating " & info.toString("repo") & "-" & _
                        result.stringify
                    Exit Function
                End If
                Debug.Print "created repo for " & info.toString("repo")
                added = added + 1
            End If
        End If
    Next info
    
    ' get them again
    If (added > 0) Then
        Set repos = getAllTheRepos(git)
        Debug.Print "added ", added, " repos"
    End If
    
    ' now add any missing readmes
    For Each info In infos.children
        Set repo = repos.findInArray("name", info.toString("repo")).parent
        
        ' check of thereis a readme and create one if noe
        Set result = git.getFileByPath(settings.toString("FILES.README"), repo)
        If (Not result.cValue("success")) Then
            Set result = writeTheFiles(git, info.toString("readmeFileId"), settings.toString("FILES.README"), repo)
        End If
        
        ' the dependencies file
        Set result = writeTheFiles(git, info.toString("dependenciesFileId"), settings.toString("FILES.DEPENDENCIES"), repo)
    
        ' the references file
        Set result = writeTheFiles(git, info.toString("crossFileId"), settings.toString("FILES.CROSS"), repo)
        
        ' the scripts
        writeTheSource git, info.kids("modules"), settings.toString("FOLDERS.SCRIPTS"), repo
        
        ' the libraries
        writeTheSource git, info.kids("dependencies"), settings.toString("FOLDERS.DEPENDENCIES"), repo
        
        ' the info file
        writeTheFiles git, info.toString("fileId"), info.toString("fileName"), repo

    Next info
    
    Set createRepos = repos

End Function
Private Function writeTheSource(git As cVbaGit, kids As Collection, _
                                    folderName As String, repo As cJobject)
    Dim job As cJobject
    For Each job In kids
        ' the source
        writeTheFiles git, _
            job.toString("id"), _
            folderName & "/" & job.toString("fileName"), _
            repo
        
        ' the docs
        writeTheFiles git, _
            job.toString("docsId"), _
            folderName & "/" & job.toString("docsName"), _
            repo
            
    Next job
    
End Function
Private Function writeTheFiles(git As cVbaGit, fileId As String, fileName As String, repo As cJobject) As cJobject
    Dim result As cJobject
    Debug.Print "committing " & fileName & " for " & repo.toString("name")
    
    Set result = git.commitFile(fileName, _
        repo, "created by vbagit", readFromFolderFile("", fileId))
        
    If (Not result.cValue("success")) Then
        MsgBox "error creating " & fileId & "-" & _
            result.stringify
    End If

    Set writeTheFiles = result
End Function
'
' these are all about reading and writing to EXTRACT.TO
'------------------------------------------------------
Private Function getAllInfoFiles(Optional specificRepoName As String = vbNullString) As cJobject

    ' get all info files in the area
    Dim infos As cJobject, settings As cJobject, s As String, a As Variant, _
        i As Long, info As cJobject
    Set infos = New cJobject
    infos.init(Nothing).addArray
    Set settings = getVGSettings()
    
    s = getAllSubFolderPaths(settings.toString("EXTRACT.TO"))
    If (s <> vbNullString) Then
        a = Split(s, ",")
        For i = LBound(a) To UBound(a)

            If fileExists(concatFolderName(CStr(a(i)), settings.toString("FILES.INFO"))) Then
                Set info = JSONParse( _
                    readFromFolderFile(CStr(a(i)), settings.toString("FILES.INFO")))
                If (specificRepoName = vbNullString Or _
                    compareAsKey(specificRepoName, info.toString("title"))) Then
                    infos.add.arrayAppend info
                Else
                    info.tearDown
                End If
            End If
            
        Next i
    End If
    
    If (specificRepoName <> vbNullString And infos.children.Count <> 1) Then
        MsgBox "didn't find repo info file for " & specificRepoName
    End If
    
    Set getAllInfoFiles = infos
End Function
Private Function writeInfoFile(project As cJobject, _
            infoJob As cJobject, _
            Optional cross As cJobject = Nothing, _
            Optional dependencyList As cJobject = Nothing) As cJobject
    Dim settings As cJobject
    Set settings = getVGSettings()
   
   ' make sure we have the directory structure set up
    checkOrCreateFolder infoJob.toString("extract")
    
    infoJob.child("modifiedDate").setValue getTimestampFromDate
    ' write it out
    writeToFolderFile "", _
        infoJob.toString("fileId"), _
        infoJob.stringify
        
    ' also need to write a readme file if there isnt one
    If (Not fileExists(infoJob.toString("readmeFileId"))) Then
        writeToFolderFile "", _
            infoJob.toString("readmeFileId"), _
            makeReadMe(infoJob)
    End If
    
    ' and a cross reference file
    If (isSomething(cross)) Then
        writeToFolderFile "", _
            infoJob.toString("crossFileId"), _
            makeCross(cross, infoJob)
    End If
    
    ' and a dependency reference file
    If (isSomething(dependencyList)) Then
        writeToFolderFile "", _
            infoJob.toString("dependenciesFileId"), _
            makeDependency(project, infoJob)
    End If


    Set writeInfoFile = infoJob

End Function
Private Function writeToStagingArea(infoJob As cJobject, dependencyList As cJobject)
    
    Dim job As cJobject, modl As cVBAmodule, code As String
    For Each job In infoJob.child("dependencies").children
        
        ' we don't write out dependencies that are already in the scripts list
        If (dependencyList.child("scripts") _
            .findInArray("name", job.toString("name")) Is Nothing) Then
            
            ' this wasnt, so its ok to go
            Set modl = dependencyList.child("dependencies") _
                .findInArray("name", job.toString("name")) _
                .parent _
                .getObject("module")
                
            If (modl.vCom.codeModule.CountOfLines > 0) Then
                code = modl.vCom.codeModule.Lines(1, modl.vCom.codeModule.CountOfLines)
            Else
                code = "'No code for this referenced module " & modl.name & vbCrLf & _
                    "'could be a problem if the reference was not for a built in excel function" & vbCrLf & _
                    "'check the cross reference md file"

                Debug.Print code
            End If
            writeToFolderFile job.toString("folder"), job.toString("fileName"), code
            
            writeToFolderFile job.toString("folder"), job.toString("docsName"), _
                makeArguments(modl, infoJob)
            
        End If
    Next job
    
    For Each job In infoJob.child("modules").children
                
        Set modl = dependencyList.child("scripts") _
            .findInArray("name", job.toString("name")) _
            .parent _
            .getObject("module")
        
        writeToFolderFile job.toString("folder"), job.toString("fileName"), _
            modl.vCom.codeModule.Lines(1, modl.vCom.codeModule.CountOfLines)

        writeToFolderFile job.toString("folder"), job.toString("docsName"), _
                makeArguments(modl, infoJob)
    Next job

End Function
'
' these are all about resolving dependencies
'-----------------------------------------------------
Private Function getDependencyList(project As cJobject, name As String, _
    Optional optListOfModules As String = vbNullString) As cJobject

    ' this is a shot at figureing out dependencies from a given list of modules
    Dim c As New cStringChunker, a As Variant, job As cJobject, dependencyList As cJobject, _
        deps As cJobject, mods As cJobject, i As Long, m As cJobject
        
    ' dependencies - all modules neededs, scripts - the ones asked for
    Set dependencyList = New cJobject
    With dependencyList.init(Nothing)
        .add "name", name
        Set deps = .add("dependencies").addArray
        Set mods = .add("scripts").addArray
    End With
    
    ' default is all modules
    If (optListOfModules = vbNullString) Then
        For Each job In project.child("modules").children
            c.add(job.toString("name")).add (",")
        Next job
        optListOfModules = c.chopIf(",").toString
    End If
    
    ' now check that they all exist
    a = Split(optListOfModules, ",")
    For i = LBound(a) To UBound(a)
        Set m = project.child("modules").findInArray("name", (CStr(a(i))))
        If (isSomething(m)) Then
            With deps.add
                .add "module", m.parent.getObject("module")
                .add "name", m.parent.getObject("module").name
            End With
            With mods.add
                .add "module", m.parent.getObject("module")
                .add "name", m.parent.getObject("module").name
            End With
        Else
            MsgBox "module doesnt exist " & CStr(a(i))
        End If
    Next i
    
    ' now we have to find modules referenced that are not in the dependecy list
    Set getDependencyList = dependencyResolve(project.child("modules"), dependencyList)
    
    
End Function
Private Function findProc(procs As Collection, targetName As String) As cVBAProcedure
    Dim proc As cVBAProcedure
    For Each proc In procs
        If (proc.name = targetName) Then
            Set findProc = proc
        End If
    Next proc
End Function
Private Function dependencyResolve(modules As cJobject, dependencyList As cJobject) As cJobject
    
    ' create a regex of all known modules that haven't yet been identified
    Dim c As cStringChunker, job As cJobject, _
        s As cStringChunker, d As cJobject, matchMod As cVBAmodule, proc As cVBAProcedure, _
        matches As MatchCollection, e As cJobject, pos As cJobject, _
        jo As cJobject, match As match, recurse As Boolean, procs As Collection, _
        code As String, pName As String, ob As Object, alreadyThere As cJobject, _
        warned As Boolean, posProc As cJobject

    Set c = New cStringChunker
    Set s = New cStringChunker

    recurse = False
    warned = False
    
    ' these are all the modules in the dependency list
    For Each job In dependencyList.child("dependencies").children
        ' get the local code
        s.clear
        Set e = New cJobject
        Set pos = New cJobject
        e.init Nothing
        pos.init(Nothing).addArray
        c.clear
        Set procs = job.getObject("module").procedures
        For Each proc In procs
            ' this is the position this code starts at - we'll need it later for finding where it came from
            
            With pos.add
                ' clean up the code getting rid of dims and continuations as well as th declaration
                code = getRidOfDims( _
                         getRidOfComments( _
                          getRidOfQuoted( _
                           Replace( _
                            straightenOutContinuations(proc.getTheCode), proc.declaration, ""))))
                
                ' remember where this code is stored
                .add "start", s.size
                .add "length", Len(code)
                .add "proc", proc
                
                ' push the code for searching
                s.add (code)
            End With
           
        Next proc
        
        ' now make a regex that describes all the other procs not in the
        ' dependency list already and not in this module
        For Each jo In modules.children
            Set matchMod = jo.getObject("module")
            If (matchMod.name <> job.toString("name")) Then
                Set d = dependencyList.child("dependencies").findInArray("name", matchMod.name)
                If (d Is Nothing) Then
                    If matchMod.textKind = "StdModule" Then
                        For Each proc In matchMod.procedures
                            ' but of course private procs will not be visible outside anyway
                            If proc.scope = "Public" And findProc(procs, proc.name) Is Nothing Then
                                e.add proc.name, proc
                                c.add(proc.name).add "|"
                            End If
                        Next proc
                    ElseIf matchMod.textKind = "ClassModule" Then
                        c.add(matchMod.name).add "|"
                        e.add matchMod.name, matchMod
                    Else
                        If (Not warned) Then
                            ' lets just tell this story one time
                            Debug.Print matchMod.textKind & " " & matchMod.name & " is skipped - only doing class and stdmodules"
                            warned = True
                        End If
                    End If
                    
                End If
            End If
        Next jo
        
        ' if we still have some to do, then kick off the matching
        If c.size > 0 Then

            'this will match all references to particular procedures
            Set matches = getRx("\b(" & c.chopIf("|").toString & ")\b(?!\s*=)").Execute(s.toString)
        
            ' we know which module they are in from the cross reference from earlier
            If (matches.Count > 0) Then
                ' add to the dependency list
                For Each match In matches
                    pName = CStr(match.SubMatches(0))
                   

                    ' find who referenced it by the position at which it appeared
                    Set posProc = getPosProc(pos, match)
                    
                    ' ob refers to the thing being called
                    Set ob = e.getObject(pName)
                    
                    ' if its a class then we dont deal in individual procs
                    If (isModuleObj(ob)) Then
                        Set matchMod = ob
                    Else
                        Set matchMod = ob.parent
                        ' but if this is a local argument to this function,
                        ' we dont need to look for references
                        ' so treat it like it was never a match
                        If (posProc.getObject("proc").isAnArgument(pName)) Then
                            Set matchMod = Nothing
                        End If
                    End If
                    
                    ' if we have found a proc/module decide if it needs to be added to depend list
                    If isSomething(matchMod) Then
                        Set alreadyThere = dependencyList.child("dependencies").findInArray("name", matchMod.name)
                        
                        ' first time we've seen it
                        If (alreadyThere Is Nothing) Then
                            Set d = dependencyList.child("dependencies").add
                            With d
                                .add "module", matchMod
                                .add "name", matchMod.name
                                If (.childExists("cross") Is Nothing) Then
                                    .add("cross").addArray
                                End If
                            End With
                            recurse = True
                        Else
                        
                        ' we already know it
                            Set d = alreadyThere.parent
                        End If
                    
                        ' record the cross reference event
                        If (alreadyThere Is Nothing Or _
                            d.child("cross").findInArray("name", _
                                    posProc.getObject("proc").name) Is Nothing) Then
                        ' first time we're seeing a reference to it so add to cross reference
                            With d.child("cross").add
                                .add "proc", ob
                                .add "by", posProc.getObject("proc")
                                .add "name", posProc.getObject("proc").name
                            End With
                        End If

                    
                    End If
                Next match
            End If
        End If
        e.tearDown
        pos.tearDown
    Next job

    ' weve added something so do it all over again
    If (recurse) Then
        dependencyResolve modules, dependencyList
    End If
    
    Set dependencyResolve = dependencyList
End Function
'/**
' * get the pos object the the procedure that provoked ths dependency
' * @param {} pos the position object for all the code of this module
' * @param {} matchOb the regex match that found this dependency
' * @return the pos object branch with the match
' */
Private Function getPosProc(pos As cJobject, matchOb As match) As cJobject
    Dim jo As cJobject
    For Each jo In pos.children
        If (matchOb.FirstIndex >= jo.cValue("start") And _
              matchOb.FirstIndex + matchOb.Length <= _
              jo.cValue("start") + jo.cValue("length")) Then
            ' this is who is referencing me
            Set getPosProc = jo
            Exit Function
        End If
    Next jo
    MsgBox ("failed to find who provoked this dependency " & matchOb)
    
End Function
Private Function makeCrossReferenceJob(dependencyList As cJobject) As cJobject
    Dim cross As cJobject, settings As cJobject, job As cJobject, jo As cJobject
    Set cross = New cJobject
    Set settings = getVGSettings()
    With cross.init(Nothing).addArray
        For Each job In dependencyList.child("dependencies").children
            If isSomething(job.childExists("cross")) Then
                For Each jo In job.children("cross").children
                    With .add
                        .add "proc", jo.getObject("proc")
                        .add "by", jo.getObject("by")
                        If (isModuleObj(jo.getObject("proc"))) Then
                            .add "sortKey", jo.getObject("proc").name
                        Else
                            .add "sortKey", jo.getObject("proc").parent.name & _
                            Space(50 - Len(jo.getObject("proc").parent.name)) & _
                            jo.getObject("proc").name
                        End If

                    End With
                Next jo
            End If
        Next job
    End With
    Set makeCrossReferenceJob = cross.sortByValue()
    
End Function
Private Sub registerExcelReferences(project As cJobject, references As cJobject)
    
    Dim job As cJobject
    
    For Each job In references.children
        registerExcelReference project, job
        
    Next job
    
End Sub
Private Function registerExcelReference(project As cJobject, job As cJobject)

    ' add a reference (if its not already there)
  
    Dim r As Reference ' Reference
    On Error GoTo handle
    With project.getObject("project").theProject
        For Each r In .references
            If (r.name = job.cValue("name")) Then
                If (r.major < job.cValue("major") Or _
                    (r.major = job.cValue("major") And _
                    r.minor < job.cValue("minor")) And Not r.BuiltIn) Then
                    .references.AddFromGuid job.cValue("guid"), _
                        job.cValue("major"), job.cValue("minor")
                    .references.remove (r)
                End If
                Exit Function
            End If
        Next r
    ' if we get here then we need to add it
      .references.AddFromGuid job.cValue("guid"), job.cValue("major"), job.cValue("minor")
      Exit Function
    End With
    
handle:
    MsgBox ("warning - tried and failed to add reference to " & _
        job.cValue("name") & "v" & job.cValue("major") _
        & "." & job.cValue("minor"))
    Exit Function
    
End Function
Private Function makeExcelReferences(project As cVBAProject, addHere As cJobject) As cJobject
                                 
    Dim r As Reference
    
    ' get all refs in this workbook
    With addHere
        For Each r In project.theProject.references
            With .add
                .add "name", r.name
                .add "guid", r.Guid
                .add "major", r.major
                .add "minor", r.minor
                .add "description", r.description
            End With
        Next r
    End With
    
    Set makeExcelReferences = addHere
End Function
'
' these are all about handling interface to VBA IDE
'-----------------------------------------------------
Private Function isModuleObj(ob As Object) As Boolean
    Dim obModel As cVBAmodule
    Set obModel = New cVBAmodule
    isModuleObj = (TypeName(ob) = TypeName(obModel))
End Function
Private Function getVbaAsJobject(Optional optProjectName As String = vbNullString) As cJobject
    Dim project As cJobject, knownProjects As cJobject, _
        module As cJobject, settings As cJobject, wb As Workbook
    
    Set settings = getVGSettings
    
    ' default is the first project that's not vbagit
    If optProjectName = vbNullString Then
        For Each wb In Workbooks
            If wb.VBProject.name <> settings.toString("PROJECT.NAME") Then
                optProjectName = wb.VBProject.name
                Exit For
            End If
        Next wb
    End If
    
    ' we must be comitting the code for vbagit
    If optProjectName = vbNullString Then
        Debug.Print "working on " & settings.toString("PROJECT.NAME"); ""
        Debug.Print "if you wanted to do one of your projects, you should have opened another workbook as well"
        optProjectName = settings.toString("PROJECT.NAME")
    End If

    ' projects in this workbook
    Set knownProjects = getProjects(optProjectName)
    
    For Each project In knownProjects.children
        
        ' get all the known modules
        For Each module In getmoduleList(project).children
            ' get all the known procedures
            getProcList module
            ' now blow out the procedures
            blowProcedures module
        Next module

    Next project
    Set getVbaAsJobject = knownProjects
End Function
Private Function blowProcedures(module As cJobject) As cJobject
    Dim pob As cVBAProcedure
    ' need to pick out to a stringifiable object
    With module.add("procedures").addArray
        For Each pob In module.getObject("module").procedures
            With .add
                .add "name", pob.name
                .add "procedure", pob
                'add the arguments
                blowArguments pob, .add("arguments").addArray
            End With

        Next pob
    End With
    Set blowProcedures = module
End Function
Private Function blowArguments(pob As cVBAProcedure, argOb As cJobject) As cJobject
    Dim argument As cVBAArgument
    ' need to pick out to a stringifiable object
    With argOb
        For Each argument In pob.arguments
            With .add
                .add "name", argument.name
                .add "argument", argument
            End With
        Next argument
    End With
    Set blowArguments = argOb
End Function

' get all projects in a workbook
Private Function getProjects(Optional optProjectName As String = vbNullString) As cJobject
    Dim wb As Workbook
    Dim project As cVBAProject
    Dim knownProjects As New cJobject
    knownProjects.init(Nothing).addArray
    
    For Each wb In Workbooks
        If wb.VBProject.name = optProjectName Or optProjectName = vbNullString Then
            Set project = New cVBAProject
            project.init wb
            With knownProjects.add
                .add "name", project.name
                .add "project", project
            End With
        End If
    Next wb
    Set getProjects = knownProjects
    
End Function
 ' get every proc in a module
 Private Sub getProcList(module As cJobject)

    Dim lStart As Long, pName As String
    Dim n As Long, s As String, t As String, doMore As Boolean, countLines As Long
    Dim cm As codeModule
    Dim pk As vbext_prockind
    Dim procedure As cVBAProcedure
    
    Set cm = module.child("module").value.vCom.codeModule
    
    lStart = cm.CountOfDeclarationLines + 1
    While lStart <= cm.CountOfLines
        pName = cm.ProcOfLine(lStart, pk)
        countLines = cm.ProcCountLines(pName, pk)
        Set procedure = New cVBAProcedure
        procedure.init module.child("module").value, pName, pk
        
        lStart = cm.ProcStartLine(pName, pk) + countLines
    Wend

End Sub
' get all modules in a project
Private Function getmoduleList(project As cJobject) As cJobject

    Dim v As VBComponent, vs As VBComponents, wb As Workbook, vj As cVBAProject
    Dim bInc As Boolean, n As Long, ml As cJobject
    
    ' this is the project object
    Set vj = project.child("project").value
    
    ' get the module components
    Set vs = vj.wBook.VBProject.VBComponents
    Dim apm As cVBAmodule
    
    n = 0
    ' add a branch to the project for modules
    Set ml = project.add("modules").addArray
    With ml
        ' append each module
        For Each v In vs
            Set apm = New cVBAmodule
            apm.init v, vj
            n = n + 1
            With .add
                .add "name", apm.name
                .add "module", apm
                .add "kind", apm.textKind
            End With
        Next v
    End With
    Set getmoduleList = ml
End Function

'
' these are all about making JSON info content
'-----------------------------------------------
Private Function makeInfoFile(project As cJobject, dependencyList As cJobject) As cJobject
    Dim infoJob As cJobject, settings As cJobject, job As cJobject
    Set infoJob = New cJobject
    Set settings = getVGSettings()
    
    ' actually the dependency list needs cut down since it contains both scripts and dependencies
    Dim library As cJobject
    Set library = New cJobject
    
    With library.init(Nothing).addArray
        For Each job In dependencyList.child("dependencies").children
            ' this means it's not in the script list
            If (dependencyList.child("scripts").findInArray("name", job.toString("name")) Is Nothing) Then
                ' since we can add objects, nothing to stop that being another cjobject!
                .add , job
            End If
        Next job
    End With

    ' now we can use the library object to driver dependencies
    With infoJob.init(Nothing)
        'the info file name .. we'll try to mirror the structur eof the google apps script/drive version
        
        ' preamble

        .add "title", dependencyList.toString("name")
        .add "committedDate", 0
        .add "createdDate", getTimestampFromDate()
        .add "modifiedDate", getTimestampFromDate()
        .add "version", settings.toString("APP.VERSION")
        .add "noticed", getTimestampFromDate()
        .add "extract", settings.toString("EXTRACT.TO") & dependencyList.toString("name") & "/"
        
        .add "fileName", settings.toString("FILES.INFO")
        .add "fileId", .toString("extract") & .toString("fileName")
        
        'module list
        modulesToInfo dependencyList.child("scripts"), _
            .add("modules").addArray, _
            .toString("extract"), _
            settings.toString("FOLDERS.SCRIPTS")

        .add "extracted", False
        .add "repo", dependencyList.toString("name")
        
        'dependency list
        modulesToInfo library, _
            .add("dependencies").addArray, _
            .toString("extract"), _
            settings.toString("FOLDERS.DEPENDENCIES")
            
        ' add excel references
        makeExcelReferences project.getObject("project"), .add("excelReferences").addArray
        
        .add "readmeFileId", .toString("extract") & settings.toString("FILES.README")
        .add "dependenciesFileId", .toString("extract") & settings.toString("FILES.DEPENDENCIES")
        .add "crossFileId", .toString("extract") & settings.toString("FILES.CROSS")

    End With
    library.tearDown
    Set makeInfoFile = infoJob
End Function

Private Function modulesToInfo(moduleJob As cJobject, infoJob As cJobject, _
        extract As String, folderName As String) As cJobject
    Dim job As cJobject, jo As cJobject, modl As cVBAmodule, fileName As String
    
    With infoJob
        For Each jo In moduleJob.children
        
            ' this is if we are using an indirection for the library
            If jo.isObjValue() Then
                Set job = jo.getObject
            Else
                Set job = jo
            End If
            
            ' get the module
            Set modl = job.getObject("module")
            fileName = modl.name & _
                    conditionalAssignment(modl.textKind = "ClassModule", ".cls", ".vba")
            With .add
                .add "name", modl.name
                .add "type", modl.textKind
                .add "folder", concatFolderName(extract, folderName) & "/"
                .add "id", concatFolderName(.toString("folder"), fileName)
                .add "fileName", fileName
                .add "docsName", modl.name & _
                    conditionalAssignment(modl.textKind = "ClassModule", "_cls", "_vba") & ".md"
                .add "docsId", concatFolderName(.toString("folder"), .toString("docsName"))
            End With
        Next jo
    End With

    Set modulesToInfo = infoJob
End Function
Private Function mdWrap()
    mdWrap = "  " & vbLf
End Function
'
'-- these are about making the content for documentation files
'--------------------------------------------------------------
Private Function makeCross(cross As cJobject, info As cJobject) As String
    Dim c As cStringChunker, job As cJobject
    Set c = New cStringChunker
    
    c.add("# VBA Project: ").addLine (info.toString("title"))
    c.add("This cross reference list for repo (").add(info.toString("repo")).add(") was automatically created on ").add(CStr(Now())).add (" by VBAGit.")
    c.addLine ("For more information see the [desktop liberation site](https://ramblings.mcpher.com/drive-sdk-and-github/getting-your-apps-scripts-to-github/ ""desktop liberation"")")
    c.add ("You can see [library and dependency information here](")
    c.add(getVGSettings().toString("FILES.DEPENDENCIES")).addLine(")").addLine ("")
    c.addLine ("###Below is a cross reference showing which modules and procedures reference which others")
    c.addLine "*module*|*proc*|*referenced by module*|*proc*"
    c.addLine "---|---|---|---"

    
    For Each job In cross.children
        
        
        ' the module being referenced
        If (isModuleObj(job.getObject("proc"))) Then
            c.add (job.getObject("proc").name)
        Else
            c.add (job.getObject("proc").parent.name)
        End If
        c.add ("|")
        
        ' the proc doing the referencing
        If (Not isModuleObj(job.getObject("proc"))) Then
            c.add (job.getObject("proc").name)
        End If
        c.add ("|")
        
        ' the module doing the referencing
        If (isModuleObj(job.getObject("by"))) Then
            c.add (job.getObject("by").name)
        Else
            c.add (job.getObject("by").parent.name)
        End If
        c.add ("|")
        
        ' the proc doing the referencing
        If (Not isModuleObj(job.getObject("by"))) Then
            c.add (job.getObject("by").name)
        End If
        c.addLine ("")
        

    Next job
    makeCross = c.toString
End Function
Private Function makeReadMe(info As cJobject) As String
    Dim c As cStringChunker
    Set c = New cStringChunker
    
    c.add("# VBA Project: ").addLine (info.toString("title"))
    c.add("This repo (").add(info.toString("repo")).add(") was automatically created on ").add(CStr(Now())).add (" by VBAGit.")
    c.addLine ("For more information see the [desktop liberation site](https://ramblings.mcpher.com/integrate-vba-with-github/ ""desktop liberation"")")
    c.add ("you can see [library and dependency information here](")
    c.add(getVGSettings().toString("FILES.DEPENDENCIES")).addLine(")").addLine ("")
    c.add ("To get started with VBA Git, you can either create a workbook with the [code on gitHub](https://github.com/brucemcpherson/VbaGit ""VbaGit repo"")")
    c.add (", or use this premade [VbaBootStrap workbook](https://ramblings.mcpher.com/wp-content/uploads/2020/02/VbaGitBootStrap.xlsm ""VbaBootStrap"")")
    c.add (mdWrap)
    c.add ("Now update manually with details of this project - this skeleton file is committed only when there is no README.md in the repo.")

    makeReadMe = c.toString
  
 
End Function

Private Function makeDependency(project As cJobject, info As cJobject) As String
    Dim c As cStringChunker, job As cJobject, settings As cJobject, jo As cJobject
    Set settings = getVGSettings(True)
    Set c = New cStringChunker
    
    c.add("# VBA Project: ").addLine (info.toString("title"))
    c.add("This repo (").add(info.toString("repo")).add (") was automatically created on ")
    c.add(CStr(Now())).add (" by VBAGit.")
    c.add ("For more information see the [desktop liberation site](https://ramblings.mcpher.com/drive-sdk-and-github/getting-your-apps-scripts-to-github/ ""desktop liberation"")")
    c.add (" or [contact me on G+](https://plus.google.com/+BruceMcpherson ""Bruce McPherson - GDE"")")
    c.add (mdWrap)
    
    c.add("## Details for VBA project ").addLine (info.toString("title"))
    c.add ("Where possibile directly referenced or sub referenced library sources have been copied to this repository")
    c.add (mdWrap)
    
    c.add("### Modules of ").add(info.toString("title")).add(" included in this repo").addLine
    c.add("*name*|*type*|*source*|*docs*").add(mdWrap).add("---|---|---|---").add (mdWrap)
   
    
    For Each job In info.kids("modules")
    
        c.add(job.toString("name")).add("|").add(job.toString("type")).add ("|")
        c.add("[").add(job.toString("fileName")).add("](").add(settings.toString("FOLDERS.SCRIPTS")).add ("/")
        c.add(job.toString("fileName")).add(" ""script source"")").add ("|")
        c.add("[").add(job.toString("docsName")).add("](").add(settings.toString("FOLDERS.SCRIPTS")).add ("/")
        c.add(job.toString("docsName")).add (" ""script docs"")")
        c.add (mdWrap)
    
    Next job
    
    c.add(mdWrap).add("### All dependencies and sub dependencies in this repo").add (mdWrap)
    c.add("*name*|*type*|*source*|*docs*").add(mdWrap).add("---|---|---|---").add (mdWrap)
    
    For Each job In info.kids("dependencies")
        c.add(job.toString("name")).add("|").add(job.toString("type")).add ("|")
        c.add("[").add(job.toString("fileName")).add("](").add(settings.toString("FOLDERS.DEPENDENCIES")).add ("/")
        c.add(job.toString("fileName")).add(" ""library source"")").add ("|")
        c.add("[").add(job.toString("docsName")).add("](").add(settings.toString("FOLDERS.DEPENDENCIES")).add ("/")
        c.add(job.toString("docsName")).add (" ""library docs"")")
        c.add (mdWrap)
    Next job

    c.add(mdWrap).add("###Excel references").add (mdWrap)
    
    If (info.child("excelReferences").children.Count > 0) Then
        c.add ("####These references were detected in the workbook (")
        c.add (project.getObject("project").wBook.name)
        c.add (") this repo was created from. You may not need them all")
        c.add (mdWrap)
        
        ' do the table titles
        For Each job In info.child("excelReferences").kids(1)
            c.add("*").add(job.key).add ("*|")
        Next job
        c.chopIf("|").add (mdWrap)
        
        For Each job In info.child("excelReferences").kids(1)
            c.add ("---|")
        Next job
        c.chopIf("|").add (mdWrap)
        
        ' now the content
        For Each jo In info.kids("excelReferences")
            For Each job In jo.children
                c.add(job.cValue).add ("|")
            Next job
            c.chopIf("|").add (mdWrap)
        Next jo
        c.chopIf("|").add (mdWrap)
        
    Else
        c.add ("####No references were detected in the workbook (")
        c.add (project.getObject("project").wBook.name)
        c.add (") this repo was created from.")
        c.add (mdWrap)
    End If
    
    c.add (mdWrap)
    c.add ("You can see [full project info as json here](")
    c.add(info.toString("fileName")).add (")")

    makeDependency = c.toString
  
End Function
Private Function constructModLink(name As String, folder As String, fileName As String, hover As String)
    Dim c As cStringChunker
    Set c = New cStringChunker
    c.add("[").add(name).add ("](")
    c.add(folder).add ("/")
    c.add(fileName).add (" """)
    If hover <> vbNullString Then
        c.add hover
    Else
        c.add name
    End If
    c.add("""").add (")")
    constructModLink = c.toString
End Function
Private Function makeArguments(modl As cVBAmodule, info As cJobject) As String
    ' this will make a mardown string for all the procedures and arguments in this module
    Dim c As cStringChunker, proc As cVBAProcedure, a As cVBAArgument
    Set c = New cStringChunker
    
    c.add("# VBA Project: **").add(info.toString("title")).addLine ("**")
    c.add("## VBA Module: **").add(findModLink(modl.name, info, "source is here", "fileName")).addLine ("**")
    c.add("### Type: ").add(modl.textKind).add("  ").addLines (2)
    c.add("This procedure list for repo (").add (info.toString("repo"))
    c.add(") was automatically created on ").add(CStr(Now())).addLine (" by VBAGit.")
    c.addLine ("For more information see the [desktop liberation site](https://ramblings.mcpher.com/drive-sdk-and-github/getting-your-apps-scripts-to-github/ ""desktop liberation"")")
    c.addLine.add("Below is a section for each procedure in ").add (modl.name)

    For Each proc In modl.procedures
        c.addLines(2).addLine ("---")
        c.add("VBA Procedure: **").add(proc.name).add("**").add("  ").addLine
        c.add("Type: **").add(proc.procTextKind).add("**").add("  ").addLine
        c.add("Returns: **").add(findModLink(proc.procReturns, info, , "docsName")).add("**").add("  ").addLine
        c.add("Return description: **").add(proc.returnDoc).add("**").add("  ").addLine
        c.add("Scope: **").add(proc.scope).add("**").add("  ").addLine
        c.add("Description: **").add(proc.description).add("**").add("  ").addLine

        c.addLine.add("*").add(proc.declaration).add("*").add("  ").addLines (2)
        
        If (proc.arguments.Count > 0) Then
            c.addLine "*name*|*type*|*optional*|*default*|*description*"
            c.addLine "---|---|---|---|---"
            
            For Each a In proc.arguments
                c.add(a.name).add ("|")
                c.add (findModLink(a.argType, info, , "docsName"))
                c.add("|").add(a.isOptional).add ("|")
                c.add(a.default).add ("|")
                c.addLine (a.description)
            Next a
        Else
            c.addLine ("**no arguments required for this procedure**")
        End If
    Next proc
    makeArguments = c.toString
End Function
Private Function findModLink(modlName As String, info As cJobject, Optional hover As String = vbNullString, _
    Optional fn As String = "docsName") As String
    
    Dim job As cJobject, settings As cJobject, c As cStringChunker
    Set c = New cStringChunker
    Set settings = getVGSettings(True)
    If hover = vbNullString Then
        hover = modlName
    End If
    For Each job In info.kids("modules")
        If (job.toString("name") = modlName) Then
            findModLink = constructModLink(modlName, "/" & settings.toString("FOLDERS.SCRIPTS"), job.toString(fn), hover)
            Exit Function
        End If
    Next job
    
    For Each job In info.kids("dependencies")
        If (job.toString("name") = modlName) Then
            findModLink = constructModLink(modlName, "/" & settings.toString("FOLDERS.DEPENDENCIES"), job.toString(fn), hover)
            Exit Function
        End If
    Next job
    
    findModLink = modlName
    
End Function
'
' these are all about handling credentials
'-----------------------------------------
Public Function getFromVbaGitRegistry(key) As String
    Dim j As cJobject
    Set j = getVGSettings().child("REGISTRY")
    getFromVbaGitRegistry = GetSetting(j.toString("root"), j.toString("app"), key)
    
End Function
Public Function setVbaGitRegistry(key, value) As String
    Dim j As cJobject
    Set j = getVGSettings().child("REGISTRY")
    SaveSetting j.toString("root"), j.toString("app"), key, value
End Function

Private Function getGitBasicCredentials()
    getGitBasicCredentials = getFromVbaGitRegistry( _
        getVGSettings().toString("REGISTRY.basic"))
End Function
Private Sub setGitBasicCredentials(user As String, pass As String)
    setVbaGitRegistry getVGSettings() _
        .toString("REGISTRY.basic"), Base64Encode(user & ":" & pass)
End Sub
Private Sub setGitClientCredentials(clientId As String, clientSecret As String)
    setVbaGitRegistry getVGSettings() _
        .toString("REGISTRY.client"), _
        Base64Encode(clientId & ":" & clientSecret)
End Sub
Private Function getGitClientCredentials()
    getGitClientCredentials = getFromVbaGitRegistry( _
        getVGSettings().toString("REGISTRY.client"))
End Function

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

527

528

529

530

531

532

533

534

535

536

537

538

539

540

541

542

543

544

545

546

547

548

549

550

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

566

567

568

569

570

571

572

573

574

575

576

577

578

579

580

581

582

583

584

585

586

587

588

589

590

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

629

630

631

632

633

634

635

636

637

638

639

640

641

642

643

644

645

646

647

648

649

650

651

652

653

654

655

656

657

658

659

660

661

662

663

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

679

680

681

682

683

684

685

686

687

688

689

690

691

692

693

694

695

696

697

698

699

700

701

702

703

704

705

706

707

708

709

710

711

712

713

714

715

716

717

718

719

720

721

722

723

724

725

726

727

728

729

730

731

732

733

734

735

736

737

738

739

740

741

742

743

744

745

746

747

748

749

750

751

752

753

754

755

756

757

758

759

760

761

762

763

764

765

766

767

768

769

770

771

772

773

774

775

776

777

778

779

780

781

782

783

784

785

786

787

788

789

790

791

792

793

794

795

796

797

798

799

800

801

802

803

804

805

806

807

808

809

810

811

812

813

814

815

816

817

818

819

820

821

822

823

824

825

826

827

828

829

830

831

832

833

834

835

836

837

838

839

840

841

842

843

844

845

846

847

848

849

850

851

852

853

854

855

856

857

858

859

860

861

862

863

864

865

866

867

868

869

870

871

872

873

874

875

876

877

878

879

880

881

882

883

884

885

886

887

888

889

890

891

892

893

894

895

896

897

898

899

900

901

902

903

904

905

906

907

908

909

910

911

912

913

914

915

916

917

918

919

920

921

922

923

924

925

926

927

928

929

930

931

932

933

934

935

936

937

938

939

940

941

942

943

944

945

946

947

948

949

950

951

952

953

954

955

956

957

958

959

960

961

962

963

964

965

966

967

968

969

970

971

972

973

974

975

976

977

978

979

980

981

982

983

984

985

986

987

988

989

990

991

992

993

994

995

996

997

998

999

1000

1001

1002

1003

1004

1005

1006

1007

1008

1009

1010

1011

1012

1013

1014

1015

1016

1017

1018

1019

1020

1021

1022

1023

1024

1025

1026

1027

1028

1029

1030

1031

1032

1033

1034

1035

1036

1037

1038

1039

1040

1041

1042

1043

1044

1045

1046

1047

1048

1049

1050

1051

1052

1053

1054

1055

1056

1057

1058

1059

1060

1061

1062

1063

1064

1065

1066

1067

1068

1069

1070

1071

1072

1073

1074

1075

1076

1077

1078

1079

1080

1081

1082

1083

1084

1085

1086

1087

1088

1089

1090

1091

1092

1093

1094

1095

1096

1097

1098

1099

1100

1101

1102

1103

1104

1105

1106

1107

1108

1109

1110

1111

1112

1113

1114

1115

1116

1117

1118

1119

1120

1121

1122

1123

1124

1125

1126

1127

1128

1129

1130

1131

1132

1133

1134

1135

1136

1137

1138

1139

1140

1141

1142

1143

1144

1145

1146

1147

1148

1149

1150

1151

1152

1153

1154

1155

1156

1157

1158

1159

1160

1161

1162

1163

1164

1165

1166

1167

1168

1169

1170

1171

1172

1173

1174

1175

1176

1177

1178

1179

1180

1181

1182

1183

1184

1185

1186

1187

1188

1189

1190

1191

1192

1193

1194

1195

1196

1197

1198

1199

1200

1201

1202

1203

1204

1205

1206

1207

1208

1209

1210

1211

1212

1213

1214

1215

1216

1217

1218

1219

1220

1221

1222

1223

1224

1225

1226

1227

1228

1229

1230

1231

1232

1233

1234

1235

1236

1237

1238

1239

1240

1241

1242

1243

1244

1245

1246

1247

1248

1249

1250

1251

1252

1253

1254

1255

1256

1257

1258

1259

1260

1261

1262

1263

1264

1265

1266

1267

1268

1269

1270

1271

1272

1273

1274

1275

1276

1277

1278

1279

1280

1281

1282

1283

1284

1285

1286

1287

1288

1289

1290

1291

1292

1293

1294

1295

1296

1297

1298

1299

1300

1301

1302

1303

1304

1305

1306

1307

1308

1309

1310

1311

1312

1313

1314

1315

1316

1317

1318

1319

1320

1321

1322

1323

1324

1325

1326

1327

1328

1329

1330

1331

1332

1333

1334

1335

1336

1337

1338

1339

1340

1341

1342

1343

1344

1345

1346

1347

1348

1349

1350

1351

1352

1353

1354

1355

1356

1357

1358

1359

1360

1361

1362

1363

1364

1365

1366

1367

1368

1369

1370

1371

1372

1373

1374

1375

1376

1377

1378

1379

1380

1381

1382

1383

1384

1385

1386

1387

1388

1389

1390

1391

1392

1393

1394

1395

1396

1397

1398

1399

1400

1401

1402

1403

1404

1405

1406

1407

1408

1409

1410

1411

1412

1413

1414

1415

1416

1417

1418

1419

1420

1421

1422

1423

1424

1425

1426

1427

1428

1429

1430

1431

1432

1433

1434

1435

1436

1437

1438

1439

1440

1441

1442

1443

1444

1445

1446

1447

1448

1449

1450

1451

1452

1453

1454

1455

1456

1457

1458

1459

1460

1461

1462

1463

1464

1465

1466

1467

1468

1469

1470

1471

1472

1473

1474

1475

1476

1477

1478

1479

1480

1481

1482

1483

1484

1485

1486

1487

1488

1489

1490

1491

1492

1493

1494

1495

1496

1497

1498

1499

1500

1501

1502

1503

1504

1505

1506

1507

1508

1509

1510

1511

1512

1513

1514

1515

1516

1517

1518

1519

1520

1521

1522

1523

1524

1525

1526

1527

1528

1529

1530

1531

1532

1533

1534

1535

Option Explicit

' this is based on the ideas from https://ramblings.mcpher.com/drive-sdk-and-github/getting-your-apps-scripts-to-github

' and is about getting your excel code to github

' VbaGit v0.2.3

' settings are in public var

Dim VGSettings As cJobject

'/**

' * example of exporting/importing a repos from github

' */

Public Sub doEverything()

doExtraction "VbaGit", "VbaGit"

doGit "VbaGit"

' these are the projects in this workbook i want to separate

'doExtraction "cJobject", "cJobject"

'doGit "cJobject"

'doExtraction "vanillacJobject", "advancedcJobject"

'doExtraction "emptycDataSet", "googleSheets,googleWireExample,oauthexamples,restLibrary"

'doGit "emptycDataSet"

' ' utilities

' doExtraction "excelClassSerializer", "classSerializer"

' ' example projects

'doExtraction "excelRestLibraryExamples", "restLibraryExamples"

' doExtraction "excelRoadmapper", "doRoadmapper"

' doExtraction "excelGoogleSheets", "googleSheets,googleWireExample"

' doExtraction "excelColor", "heatmapExamples,colorizing"

' doExtraction "excelD3", "D3"

' doExtraction "excelOauth2", "oAuthExamples"

' doExtraction "excelParseCom", "parseCom"

' doExtraction "excelProgressBar", "TestProgressBar"

' doExtraction "cChromeTraceVBA", "cChromeTraceVBA,testChromeTrace"

'doExtraction "cVBAProject", "cVBAProject,cVBAProcedure,cVBAmodule,cVBAArgument"

'doExtraction "cDataSet", "cDataSet"

'doExtraction "excelRestLibrary", "restLibrary,cRest"

' now write them to git

'doGit "emptycDataSet"

'doExtraction "effex-demo-markers-excel", "effexTests,VBAMapsEffex"

'doGit "effex-demo-markers-excel"

End Sub

'/**

' * example of importing a repo from github and replaces the code in the companion wokbook

' */

Public Sub doTheImport()

' this is the something I want to import into the companion workbook

'//doImportFromGit "cDataSet"

''doImportFromGit "cJobject"

End Sub

' NOTES ON IMPORTING CODE FROM GITHUB

' since this is for public code,

' there is no need to register or create a github app (unlike for writing code)

' although if you do have credentials you can apply them if you want

' 1 .open the vbagit worksheet and the sheet to which you want to import the code

' 2 .set up do the import

' first argument is the repoName to get the code from

' second is the project name - you should probably leave that blank

' third is whether to apply the excel references - the defaut is not to

' 3. run it and compile the result

' ----------------------------------------

' NOTES ON COMMITTING CODE TO GITHUB

' open the vbagit workbook and set up for your environment if you havent already done it

' 1. set your credentials in deleteThisAfterRunningOnce,

' then either delete or obscure your credentials, you wont need to run this again

' you need to have set up an app in github and got the api credentials

' 2. set getVGSettings for your environment

' this describes who you are and where to do things

' the main ones are

' EXTRACT.TO (a staging area folder for all your files to be dumped)

' GIT.COMMITTER

' GIT.USERAGENT

' you probably don't need to change any others unless you want to rename or reorganize the repo contents

' vbagit runs as a companion workbook

' you open vbagit

' you open the workbook containing the modules you want to commit

' switch to the vbagit workbook and set up what you want to do

' 3. run doextraction for each repo you want to create, naming the module(s) for the repo

' you can specify a project name if you have more than 2 workbooks open (vbagit + the one containing the modules to be processed)

' however most people leave their projects with the default name of VBAProject.

' It will take the first one it finds with the given name - so its best just to have the right workbook open

' the first arg wil be the reponame - this what it will be called on GIThub

' the second is the list of modules that are to be part of this repo

' you just need to name the main module(s). These will end up in the scripts folder

' Any dependent modules/classes will be automatically detected and added to the libraries folder

' Module Documentation is also automatically generated at this stage

' Note that you can create multiple repos from a single workbook

' see doEverything for an example

' any shared libraries needed will be detected and committed to whichever repo as required

' 4. when ready you can run doGit to commit everything in the EXTRACT.TO folder up to GIT

' actually, its the contents of info.json that decides what to copy up. anyting in EXTRACT.TO not created by

' vbagit will be ignored

' all repos will be commited at once, except if you specify a repoName to doGit. then it will only do one

' your readme will only be committed if there is not already one in the Github repo

' if you prefer to use the git client instead, you can make your EXTRACT.TO location your local git repo

'/**

' * sets up your credentials in the windows registry.

' * should be deleted or obfuscated after running once

' */

Private Function deleteThisAfterRunningOnce()

' substitute your git application clientid/secret

setGitBasicCredentials "username git", "passwrod git"

setGitClientCredentials "short", "long"

End Function

'/**

' * sets up the settings object if its not already set up and returns it

' * @param {boolean} force whether to force a new set up

' * @return {cJobject} the settings

' */

Public Function getVGSettings(Optional force As Boolean)

' get the settings - only bothers with the parse once

If force Or isUndefined(VGSettings) Then

If (isSomething(VGSettings)) Then

VGSettings.tearDown

End If

Set VGSettings = New cJobject

With VGSettings.init(Nothing)

With .add("EXTRACT")

.add "TO", "c:/users/bruce/documents/gas/Extraction/Scripts/"

End With

With .add("GIT")

With .add("COMMITTER")

.add "name", "Bruce McPherson"

.add "email", "bruce@mcpher.com"

End With

.add "USERAGENT", "brucemcpherson"

.add "SCOPES", "repo,gist"

.add "OWNER", .toString("USERAGENT")

End With

With .add("REGISTRY")

.add "root", "xLiberation"

.add "app", "vbagit"

.add "basic", "basichash"

.add "client", "clienthash"

End With

With .add("APP")

.add "VERSION", "0.2.4"

End With

With .add("FILES")

.add "README", "README.md"

.add "INFO", "info.json"

.add "DEPENDENCIES", "dependencies.md"

.add "CROSS", "cross.md"

End With

With .add("FOLDERS")

.add "SCRIPTS", "scripts"

.add "DEPENDENCIES", "libraries"

End With

With .add("PROJECT")

.add "NAME", "VbaGitAddOn"

End With

With .add("VBA")

With .add("TYPES")

.add "StdModule", 1

.add "ClassModule", 2

End With

End If

Set getVGSettings = VGSettings

End Function

'/**

' * do the import from github and replace the modules in the companion workbook

' * @param {} repoName the github reponame

' * @param {} projectName the vbaproject name

' * @param {} applyExcelReferences whether to apply the excel references in dependency list

' */

Public Sub doImportFromGit(repoName As String, _

Optional projectName As String = vbNullString, _

Optional applyExcelReferences As Boolean = False)

' get all the projects in this workbook

Dim projects As cJobject, settings As cJobject, project As cJobject

Dim repo As cJobject, git As cVbaGit, result As cJobject, _

job As cJobject, info As cJobject

Set settings = getVGSettings(True)

Set projects = getVbaAsJobject(projectName)

Set project = projects.children(1)

' create dependency list - only do the first project for this scope

If (project.getObject("project").name = settings.toString("PROJECT.NAME")) Then

MsgBox "you need to open both vbagit and the workbook to set up: you cannot overwrite vbagit"

Exit Sub

End If

' check we are doing the right thing

If MsgBox("You are 100% sure that you want me to import the code from repo " & repoName & _

" from github into " & project.getObject("project").wBook.name, vbYesNo) <> vbYes Then

Exit Sub

Else

Debug.Print "Importing the code from repo " & repoName & _

" from github into " & project.getObject("project").wBook.name

' get a handle for git api

Set git = New cVbaGit

' actually we are using basic authentication

' this optional, if you have already setup a github app

' it will also get you more quota

' if not then you can comment this out

git.setAccessToken getGitBasicCredentials(), getGitClientCredentials()

If (Not git.isAccessToken) Then

Debug.Print "you are using an unauthenticated git connection with limited quota"

End If

' get the repo

Set repo = getRepo(git, repoName).getObject("data")

' get the info file

Set result = git.getFileByPath(settings.toString("FILES.INFO"), repo)

' process the scripts

Set info = JSONParse(result.toString("content"))

' the modules

getCodeFromGit project, git, _

settings.toString("FOLDERS.SCRIPTS"), info, _

"modules", repo

' the libraries

getCodeFromGit project, git, _

settings.toString("FOLDERS.DEPENDENCIES"), info, _

"dependencies", repo

If (applyExcelReferences) Then

Debug.Print "Apply excel references"

registerExcelReferences project, info.child("excelReferences")

End If

info.tearDown

End If

End Sub

'/**

' * get the code from git for a particular module

' * @param {} project the project object

' * @param {} git a handle to the cVbaGit object

' * @param {} folder the folder to find the file in

' * @param {} childName the branch of the project to work from (scripts/libraries)

' * @param {} repo the repo object containing this file

' */

Private Sub getCodeFromGit(project As cJobject, git As cVbaGit, _

folder As String, info As cJobject, _

childName As String, repo As cJobject)

Dim job As cJobject, result As cJobject

Debug.Print "Importing project " & childName

For Each job In info.kids(childName)

Set result = git.getFileByPath(folder & "/" & job.toString("fileName"), repo)

Debug.Assert result.cValue("success")

replaceModule project, job, result.toString("content")

result.tearDown

Next job

End Sub

'/**

' * get the code from git for a particular module

' * @param {} project the project object

' * @param {} infoItem the object from info.json for this file

' * @param {} code the new code to use

' * @return {} whether it was successful

' */

Private Function replaceModule(project As cJobject, infoItem As cJobject, code As String) As Boolean

Dim jm As cJobject, module As VBComponent, m As cJobject, _

t As Long, settings As cJobject, vm As cVBAmodule

Set settings = getVGSettings

Set m = project.child("modules").findInArray("name", infoItem.toString("name"))

If (m Is Nothing) Then

' we need to create a new module

Debug.Print "creating "; infoItem.toString("type"); " "; infoItem.toString("name")

Set module = project.getObject("project") _

.theProject _

.VBComponents _

.add(settings.cValue("VBA.TYPES." & infoItem.toString("type")))

module.name = infoItem.toString("name")

Else

Set vm = m.parent.getObject("module")

Set module = vm.vCom

If (vm.textKind <> infoItem.toString("type")) Then

Debug.Print "module " & vm.name & " is not the same type as on github: cannot replace it"

Exit Function

End If

' delete existing content

Debug.Print "replacing "; infoItem.toString("type"); " "; infoItem.toString("name")

End If

'clear current contents

If module.codeModule.CountOfLines > 0 Then

' remove Option Explict lines if it was added automatically

module.codeModule.DeleteLines 1, module.codeModule.CountOfLines

End If

' add the new code

module.codeModule.AddFromString code

replaceModule = True

End Function

'/**

' * get the code from git for a particular module

' * @param {} git a cVbaGit handle

' * @param {} repoName the name of the repo

' * @param {} complain whether to complain on failure

' * @return {} the repo object

' */

Private Function getRepo(git As cVbaGit, repoName As String, Optional complain As Boolean = True) As cJobject

Dim settings As cJobject, result As cJobject

Set settings = getVGSettings()

Set result = git.getSpecificRepo(settings.toString("GIT.OWNER"), repoName)

If (complain And Not result.cValue("success")) Then

Err.Raise vbObjectError + result.cValue("code"), result.stringify

End If

Set getRepo = result

End Function

'/**

' * extract the files for a particular project and write them to the staging area

' * @param {} repoName the name of the repo

' * @param {} optListOfModules list of main modules to use as starting point

' * @param {} projectName the name of the vba project

' */

Private Sub doExtraction(repoName As String, _

Optional optListOfModules As String = vbNullString, _

Optional projectName As String = vbNullString)

' get all the projects in this workbook

Dim projects As cJobject, dependencyList As cJobject, _

infoJob As cJobject, settings As cJobject, project As cJobject

Set settings = getVGSettings(True)

Set projects = getVbaAsJobject(projectName)

' create dependency list - only do the first project for this scope

If (projects.hasChildren) Then

Set project = projects.children(1)

Set dependencyList = getDependencyList(project, repoName, optListOfModules)

Else

' no projects

MsgBox "no projects detected for repo " & repoName

Exit Sub

End If

Debug.Print "extracting repo "; repoName; " from project "; project.getObject("project").theProject.name

' create the info & cross references file

Set infoJob = makeInfoFile(project, dependencyList)

' now we write all the scripts to some staging area

writeToStagingArea infoJob, dependencyList

' mark as extracted

infoJob.child("extracted").setValue True

writeInfoFile project, infoJob, makeCrossReferenceJob(dependencyList), dependencyList

' clean up

projects.tearDown

dependencyList.tearDown

infoJob.tearDown

settings.tearDown

Debug.Print "done extracting "; repoName

End Sub

Private Sub testmodulestuff()

Dim job As cJobject, settings As cJobject, projects As cJobject, proc As cVBAProcedure

Dim r As Range, rx As RegExp

Set r = Range("sheet1!a1")

Set settings = getVGSettings(True)

Set projects = getVbaAsJobject()

For Each job In projects.children(1).kids("modules")

With job.getObject("module")

For Each proc In .procedures

With proc

Set r = r.Offset(1)

r.Offset(, 0).value = .name

r.Offset(, 1).value = .startLine

r.Offset(, 2).value = .lineCount

r.Offset(, 3).value = .getTheEndRx.Test( _

.codeModule.Lines(.startLine, _

.getFinishWithoutTrailingComments - .startLine + 1))

r.Offset(, 4).value = .getTheCode

r.Offset(, 5).value = .getTheCodePlusLeadingComments

End With

Next proc

End With

Next job

projects.tearDown

End Sub

' these are all about committing to Git

'--------------------------------------------------------

'/**

' * call this to commit all extracted projects to github them from the staging area

' * @param {} specificRepoName the name of the repo - if blank it will do them all

' */

Private Sub doGit(Optional specificRepoName As String = vbNullString)

Dim allInfoFiles As cJobject, settings As cJobject, git As cVbaGit, _

repos As cJobject, result As cJobject

Set settings = getVGSettings(True)

' the info files drive what needs to be written to Git

Set allInfoFiles = getAllInfoFiles(specificRepoName)

' get a handle for git api

Set git = New cVbaGit

' actually we are using basic authentication

git.setAccessToken getGitBasicCredentials(), getGitClientCredentials()

If (Not git.isAccessToken) Then

MsgBox ("you cannot commit to git without authentication- please set up")

Else

' get all the repos, creating any missing ones, and adding all the files

Set repos = createRepos(git, allInfoFiles)

End If

' clean up

allInfoFiles.tearDown

repos.tearDown

git.tearDown

End Sub

'/**

' * get all known repos belonging to the git logged in individual

' * @param {} git a handle to the cVbaGit api

' * @return {} all the known repos

' */

Private Function getAllTheRepos(git As cVbaGit) As cJobject

Dim result As cJobject

Set result = git.getMyRepos

If (result.cValue("success")) Then

Set getAllTheRepos = result.getObject("data")

Else

MsgBox "failed to get all the repos " + result.stringify

Exit Function

End If

End Function

'/**

' * create any repos in our list of info objects that don't exist

' * @param {} git a handle to the cVbaGit api

' * @param {} infos a list of info objects

' * @return {} all the known repos updated

' */

Private Function createRepos(git As cVbaGit, infos As cJobject) As cJobject

Dim repos As cJobject, info As cJobject, repo As cJobject, result As cJobject, _

added As Long, settings As cJobject, job As cJobject

Set settings = getVGSettings

' all my repos

Set repos = getAllTheRepos(git)

added = 0

' find any missing

For Each info In infos.children

' we'll only do uncommitted or modified since last commit

If (info.cValue("committedDate") < info.cValue("modifieddate")) Then

If (isSomething(repos)) Then

Set repo = repos.findInArray("name", info.toString("repo"))

Else

Set repo = Nothing

End If

' need to create it

If (repo Is Nothing) Then

Set result = git.createRepo(info.toString("repo"))

If (Not result.cValue("success")) Then

MsgBox "error creating " & info.toString("repo") & "-" & _

result.stringify

Exit Function

End If

Debug.Print "created repo for " & info.toString("repo")

added = added + 1

End If

Next info

' get them again

If (added > 0) Then

Set repos = getAllTheRepos(git)

Debug.Print "added ", added, " repos"

End If

' now add any missing readmes

For Each info In infos.children

Set repo = repos.findInArray("name", info.toString("repo")).parent

' check of thereis a readme and create one if noe

Set result = git.getFileByPath(settings.toString("FILES.README"), repo)

If (Not result.cValue("success")) Then

Set result = writeTheFiles(git, info.toString("readmeFileId"), settings.toString("FILES.README"), repo)

End If

' the dependencies file

Set result = writeTheFiles(git, info.toString("dependenciesFileId"), settings.toString("FILES.DEPENDENCIES"), repo)

' the references file

Set result = writeTheFiles(git, info.toString("crossFileId"), settings.toString("FILES.CROSS"), repo)

' the scripts

writeTheSource git, info.kids("modules"), settings.toString("FOLDERS.SCRIPTS"), repo

' the libraries

writeTheSource git, info.kids("dependencies"), settings.toString("FOLDERS.DEPENDENCIES"), repo

' the info file

writeTheFiles git, info.toString("fileId"), info.toString("fileName"), repo

Next info

Set createRepos = repos

End Function

Private Function writeTheSource(git As cVbaGit, kids As Collection, _

folderName As String, repo As cJobject)

Dim job As cJobject

For Each job In kids

' the source

writeTheFiles git, _

job.toString("id"), _

folderName & "/" & job.toString("fileName"), _

repo

' the docs

writeTheFiles git, _

job.toString("docsId"), _

folderName & "/" & job.toString("docsName"), _

repo

Next job

End Function

Private Function writeTheFiles(git As cVbaGit, fileId As String, fileName As String, repo As cJobject) As cJobject

Dim result As cJobject

Debug.Print "committing " & fileName & " for " & repo.toString("name")

Set result = git.commitFile(fileName, _

repo, "created by vbagit", readFromFolderFile("", fileId))

If (Not result.cValue("success")) Then

MsgBox "error creating " & fileId & "-" & _

result.stringify

End If

Set writeTheFiles = result

End Function

' these are all about reading and writing to EXTRACT.TO

'------------------------------------------------------

Private Function getAllInfoFiles(Optional specificRepoName As String = vbNullString) As cJobject

' get all info files in the area

Dim infos As cJobject, settings As cJobject, s As String, a As Variant, _

i As Long, info As cJobject

Set infos = New cJobject

infos.init(Nothing).addArray

Set settings = getVGSettings()

s = getAllSubFolderPaths(settings.toString("EXTRACT.TO"))

If (s <> vbNullString) Then

a = Split(s, ",")

For i = LBound(a) To UBound(a)

If fileExists(concatFolderName(CStr(a(i)), settings.toString("FILES.INFO"))) Then

Set info = JSONParse( _

readFromFolderFile(CStr(a(i)), settings.toString("FILES.INFO")))

If (specificRepoName = vbNullString Or _

compareAsKey(specificRepoName, info.toString("title"))) Then

infos.add.arrayAppend info

Else

info.tearDown

End If

Next i

End If

If (specificRepoName <> vbNullString And infos.children.Count <> 1) Then

MsgBox "didn't find repo info file for " & specificRepoName

End If

Set getAllInfoFiles = infos

End Function

Private Function writeInfoFile(project As cJobject, _

infoJob As cJobject, _

Optional cross As cJobject = Nothing, _

Optional dependencyList As cJobject = Nothing) As cJobject

Dim settings As cJobject

Set settings = getVGSettings()

' make sure we have the directory structure set up

checkOrCreateFolder infoJob.toString("extract")

infoJob.child("modifiedDate").setValue getTimestampFromDate

' write it out

writeToFolderFile "", _

infoJob.toString("fileId"), _

infoJob.stringify

' also need to write a readme file if there isnt one

If (Not fileExists(infoJob.toString("readmeFileId"))) Then

writeToFolderFile "", _

infoJob.toString("readmeFileId"), _

makeReadMe(infoJob)

End If

' and a cross reference file

If (isSomething(cross)) Then

writeToFolderFile "", _

infoJob.toString("crossFileId"), _

makeCross(cross, infoJob)

End If

' and a dependency reference file

If (isSomething(dependencyList)) Then

writeToFolderFile "", _

infoJob.toString("dependenciesFileId"), _

makeDependency(project, infoJob)

End If

Set writeInfoFile = infoJob

End Function

Private Function writeToStagingArea(infoJob As cJobject, dependencyList As cJobject)

Dim job As cJobject, modl As cVBAmodule, code As String

For Each job In infoJob.child("dependencies").children

' we don't write out dependencies that are already in the scripts list

If (dependencyList.child("scripts") _

.findInArray("name", job.toString("name")) Is Nothing) Then

' this wasnt, so its ok to go

Set modl = dependencyList.child("dependencies") _

.findInArray("name", job.toString("name")) _

.parent _

.getObject("module")

If (modl.vCom.codeModule.CountOfLines > 0) Then

code = modl.vCom.codeModule.Lines(1, modl.vCom.codeModule.CountOfLines)

Else

code = "'No code for this referenced module " & modl.name & vbCrLf & _

"'could be a problem if the reference was not for a built in excel function" & vbCrLf & _

"'check the cross reference md file"

Debug.Print code

End If

writeToFolderFile job.toString("folder"), job.toString("fileName"), code

writeToFolderFile job.toString("folder"), job.toString("docsName"), _

makeArguments(modl, infoJob)

End If

Next job

For Each job In infoJob.child("modules").children

Set modl = dependencyList.child("scripts") _

.findInArray("name", job.toString("name")) _

.parent _

.getObject("module")

writeToFolderFile job.toString("folder"), job.toString("fileName"), _

modl.vCom.codeModule.Lines(1, modl.vCom.codeModule.CountOfLines)

writeToFolderFile job.toString("folder"), job.toString("docsName"), _

makeArguments(modl, infoJob)

Next job

End Function

' these are all about resolving dependencies

'-----------------------------------------------------

Private Function getDependencyList(project As cJobject, name As String, _

Optional optListOfModules As String = vbNullString) As cJobject

' this is a shot at figureing out dependencies from a given list of modules

Dim c As New cStringChunker, a As Variant, job As cJobject, dependencyList As cJobject, _

deps As cJobject, mods As cJobject, i As Long, m As cJobject

' dependencies - all modules neededs, scripts - the ones asked for

Set dependencyList = New cJobject

With dependencyList.init(Nothing)

.add "name", name

Set deps = .add("dependencies").addArray

Set mods = .add("scripts").addArray

End With

' default is all modules

If (optListOfModules = vbNullString) Then

For Each job In project.child("modules").children

c.add(job.toString("name")).add (",")

Next job

optListOfModules = c.chopIf(",").toString

End If

' now check that they all exist

a = Split(optListOfModules, ",")

For i = LBound(a) To UBound(a)

Set m = project.child("modules").findInArray("name", (CStr(a(i))))

If (isSomething(m)) Then

With deps.add

.add "module", m.parent.getObject("module")

.add "name", m.parent.getObject("module").name

End With

With mods.add

.add "module", m.parent.getObject("module")

.add "name", m.parent.getObject("module").name

End With

Else

MsgBox "module doesnt exist " & CStr(a(i))

End If

Next i

' now we have to find modules referenced that are not in the dependecy list

Set getDependencyList = dependencyResolve(project.child("modules"), dependencyList)

End Function

Private Function findProc(procs As Collection, targetName As String) As cVBAProcedure

Dim proc As cVBAProcedure

For Each proc In procs

If (proc.name = targetName) Then

Set findProc = proc

End If

Next proc

End Function

Private Function dependencyResolve(modules As cJobject, dependencyList As cJobject) As cJobject

' create a regex of all known modules that haven't yet been identified

Dim c As cStringChunker, job As cJobject, _

s As cStringChunker, d As cJobject, matchMod As cVBAmodule, proc As cVBAProcedure, _

matches As MatchCollection, e As cJobject, pos As cJobject, _

jo As cJobject, match As match, recurse As Boolean, procs As Collection, _

code As String, pName As String, ob As Object, alreadyThere As cJobject, _

warned As Boolean, posProc As cJobject

Set c = New cStringChunker

Set s = New cStringChunker

recurse = False

warned = False

' these are all the modules in the dependency list

For Each job In dependencyList.child("dependencies").children

' get the local code

s.clear

Set e = New cJobject

Set pos = New cJobject

e.init Nothing

pos.init(Nothing).addArray

c.clear

Set procs = job.getObject("module").procedures

For Each proc In procs

' this is the position this code starts at - we'll need it later for finding where it came from

With pos.add

' clean up the code getting rid of dims and continuations as well as th declaration

code = getRidOfDims( _

getRidOfComments( _

getRidOfQuoted( _

Replace( _

straightenOutContinuations(proc.getTheCode), proc.declaration, ""))))

' remember where this code is stored

.add "start", s.size

.add "length", Len(code)

.add "proc", proc

' push the code for searching

s.add (code)

End With

Next proc

' now make a regex that describes all the other procs not in the

' dependency list already and not in this module

For Each jo In modules.children

Set matchMod = jo.getObject("module")

If (matchMod.name <> job.toString("name")) Then

Set d = dependencyList.child("dependencies").findInArray("name", matchMod.name)

If (d Is Nothing) Then

If matchMod.textKind = "StdModule" Then

For Each proc In matchMod.procedures

' but of course private procs will not be visible outside anyway

If proc.scope = "Public" And findProc(procs, proc.name) Is Nothing Then

e.add proc.name, proc

c.add(proc.name).add "|"

End If

Next proc

ElseIf matchMod.textKind = "ClassModule" Then

c.add(matchMod.name).add "|"

e.add matchMod.name, matchMod

Else

If (Not warned) Then

' lets just tell this story one time

Debug.Print matchMod.textKind & " " & matchMod.name & " is skipped - only doing class and stdmodules"

warned = True

End If

Next jo

' if we still have some to do, then kick off the matching

If c.size > 0 Then

'this will match all references to particular procedures

Set matches = getRx("\b(" & c.chopIf("|").toString & ")\b(?!\s*=)").Execute(s.toString)

' we know which module they are in from the cross reference from earlier

If (matches.Count > 0) Then

' add to the dependency list

For Each match In matches

pName = CStr(match.SubMatches(0))

' find who referenced it by the position at which it appeared

Set posProc = getPosProc(pos, match)

' ob refers to the thing being called

Set ob = e.getObject(pName)

' if its a class then we dont deal in individual procs

If (isModuleObj(ob)) Then

Set matchMod = ob

Else

Set matchMod = ob.parent

' but if this is a local argument to this function,

' we dont need to look for references

' so treat it like it was never a match

If (posProc.getObject("proc").isAnArgument(pName)) Then

Set matchMod = Nothing

End If

' if we have found a proc/module decide if it needs to be added to depend list

If isSomething(matchMod) Then

Set alreadyThere = dependencyList.child("dependencies").findInArray("name", matchMod.name)

' first time we've seen it

If (alreadyThere Is Nothing) Then

Set d = dependencyList.child("dependencies").add

With d

.add "module", matchMod

.add "name", matchMod.name

If (.childExists("cross") Is Nothing) Then

.add("cross").addArray

End If

End With

recurse = True

Else

' we already know it

Set d = alreadyThere.parent

End If

' record the cross reference event

If (alreadyThere Is Nothing Or _

d.child("cross").findInArray("name", _

posProc.getObject("proc").name) Is Nothing) Then

' first time we're seeing a reference to it so add to cross reference

With d.child("cross").add

.add "proc", ob

.add "by", posProc.getObject("proc")

.add "name", posProc.getObject("proc").name

End With

End If

Next match

End If

e.tearDown

pos.tearDown

Next job

' weve added something so do it all over again

If (recurse) Then

dependencyResolve modules, dependencyList

End If

Set dependencyResolve = dependencyList

End Function

'/**

' * get the pos object the the procedure that provoked ths dependency

' * @param {} pos the position object for all the code of this module

' * @param {} matchOb the regex match that found this dependency

' * @return the pos object branch with the match

' */

Private Function getPosProc(pos As cJobject, matchOb As match) As cJobject

Dim jo As cJobject

For Each jo In pos.children

If (matchOb.FirstIndex >= jo.cValue("start") And _

matchOb.FirstIndex + matchOb.Length <= _

jo.cValue("start") + jo.cValue("length")) Then

' this is who is referencing me

Set getPosProc = jo

Exit Function

End If

Next jo

MsgBox ("failed to find who provoked this dependency " & matchOb)

End Function

Private Function makeCrossReferenceJob(dependencyList As cJobject) As cJobject

Dim cross As cJobject, settings As cJobject, job As cJobject, jo As cJobject

Set cross = New cJobject

Set settings = getVGSettings()

With cross.init(Nothing).addArray

For Each job In dependencyList.child("dependencies").children

If isSomething(job.childExists("cross")) Then

For Each jo In job.children("cross").children

With .add

.add "proc", jo.getObject("proc")

.add "by", jo.getObject("by")

If (isModuleObj(jo.getObject("proc"))) Then

.add "sortKey", jo.getObject("proc").name

Else

.add "sortKey", jo.getObject("proc").parent.name & _

Space(50 - Len(jo.getObject("proc").parent.name)) & _

jo.getObject("proc").name

End If

End With

Next jo

End If

Next job

End With

Set makeCrossReferenceJob = cross.sortByValue()

End Function

Private Sub registerExcelReferences(project As cJobject, references As cJobject)

Dim job As cJobject

For Each job In references.children

registerExcelReference project, job

Next job

End Sub

Private Function registerExcelReference(project As cJobject, job As cJobject)

' add a reference (if its not already there)

Dim r As Reference ' Reference

On Error GoTo handle

With project.getObject("project").theProject

For Each r In .references

If (r.name = job.cValue("name")) Then

If (r.major < job.cValue("major") Or _

(r.major = job.cValue("major") And _

r.minor < job.cValue("minor")) And Not r.BuiltIn) Then

.references.AddFromGuid job.cValue("guid"), _

job.cValue("major"), job.cValue("minor")

.references.remove (r)

End If

Exit Function

End If

Next r

' if we get here then we need to add it

.references.AddFromGuid job.cValue("guid"), job.cValue("major"), job.cValue("minor")

Exit Function

End With

handle:

MsgBox ("warning - tried and failed to add reference to " & _

job.cValue("name") & "v" & job.cValue("major") _

& "." & job.cValue("minor"))

Exit Function

End Function

Private Function makeExcelReferences(project As cVBAProject, addHere As cJobject) As cJobject

Dim r As Reference

' get all refs in this workbook

With addHere

For Each r In project.theProject.references

With .add

.add "name", r.name

.add "guid", r.Guid

.add "major", r.major

.add "minor", r.minor

.add "description", r.description

End With

Next r

End With

Set makeExcelReferences = addHere

End Function

' these are all about handling interface to VBA IDE

'-----------------------------------------------------

Private Function isModuleObj(ob As Object) As Boolean

Dim obModel As cVBAmodule

Set obModel = New cVBAmodule

isModuleObj = (TypeName(ob) = TypeName(obModel))

End Function

Private Function getVbaAsJobject(Optional optProjectName As String = vbNullString) As cJobject

Dim project As cJobject, knownProjects As cJobject, _

module As cJobject, settings As cJobject, wb As Workbook

Set settings = getVGSettings

' default is the first project that's not vbagit

If optProjectName = vbNullString Then

For Each wb In Workbooks

If wb.VBProject.name <> settings.toString("PROJECT.NAME") Then

optProjectName = wb.VBProject.name

Exit For

End If

Next wb

End If

' we must be comitting the code for vbagit

If optProjectName = vbNullString Then

Debug.Print "working on " & settings.toString("PROJECT.NAME"); ""

Debug.Print "if you wanted to do one of your projects, you should have opened another workbook as well"

optProjectName = settings.toString("PROJECT.NAME")

End If

' projects in this workbook

Set knownProjects = getProjects(optProjectName)

For Each project In knownProjects.children

' get all the known modules

For Each module In getmoduleList(project).children

' get all the known procedures

getProcList module

' now blow out the procedures

blowProcedures module

Next module

Next project

Set getVbaAsJobject = knownProjects

End Function

Private Function blowProcedures(module As cJobject) As cJobject

Dim pob As cVBAProcedure

' need to pick out to a stringifiable object

With module.add("procedures").addArray

For Each pob In module.getObject("module").procedures

With .add

.add "name", pob.name

.add "procedure", pob

'add the arguments

blowArguments pob, .add("arguments").addArray

End With

Next pob

End With

Set blowProcedures = module

End Function

Private Function blowArguments(pob As cVBAProcedure, argOb As cJobject) As cJobject

Dim argument As cVBAArgument

' need to pick out to a stringifiable object

With argOb

For Each argument In pob.arguments

With .add

.add "name", argument.name

.add "argument", argument

End With

Next argument

End With

Set blowArguments = argOb

End Function

' get all projects in a workbook

Private Function getProjects(Optional optProjectName As String = vbNullString) As cJobject

Dim wb As Workbook

Dim project As cVBAProject

Dim knownProjects As New cJobject

knownProjects.init(Nothing).addArray

For Each wb In Workbooks

If wb.VBProject.name = optProjectName Or optProjectName = vbNullString Then

Set project = New cVBAProject

project.init wb

With knownProjects.add

.add "name", project.name

.add "project", project

End With

End If

Next wb

Set getProjects = knownProjects

End Function

' get every proc in a module

Private Sub getProcList(module As cJobject)

Dim lStart As Long, pName As String

Dim n As Long, s As String, t As String, doMore As Boolean, countLines As Long

Dim cm As codeModule

Dim pk As vbext_prockind

Dim procedure As cVBAProcedure

Set cm = module.child("module").value.vCom.codeModule

lStart = cm.CountOfDeclarationLines + 1

While lStart <= cm.CountOfLines

pName = cm.ProcOfLine(lStart, pk)

countLines = cm.ProcCountLines(pName, pk)

Set procedure = New cVBAProcedure

procedure.init module.child("module").value, pName, pk

lStart = cm.ProcStartLine(pName, pk) + countLines

Wend

End Sub

' get all modules in a project

Private Function getmoduleList(project As cJobject) As cJobject

Dim v As VBComponent, vs As VBComponents, wb As Workbook, vj As cVBAProject

Dim bInc As Boolean, n As Long, ml As cJobject

' this is the project object

Set vj = project.child("project").value

' get the module components

Set vs = vj.wBook.VBProject.VBComponents

Dim apm As cVBAmodule

n = 0

' add a branch to the project for modules

Set ml = project.add("modules").addArray

With ml

' append each module

For Each v In vs

Set apm = New cVBAmodule

apm.init v, vj

n = n + 1

With .add

.add "name", apm.name

.add "module", apm

.add "kind", apm.textKind

End With

Next v

End With

Set getmoduleList = ml

End Function

' these are all about making JSON info content

'-----------------------------------------------

Private Function makeInfoFile(project As cJobject, dependencyList As cJobject) As cJobject

Dim infoJob As cJobject, settings As cJobject, job As cJobject

Set infoJob = New cJobject

Set settings = getVGSettings()

' actually the dependency list needs cut down since it contains both scripts and dependencies

Dim library As cJobject

Set library = New cJobject

With library.init(Nothing).addArray

For Each job In dependencyList.child("dependencies").children

' this means it's not in the script list

If (dependencyList.child("scripts").findInArray("name", job.toString("name")) Is Nothing) Then

' since we can add objects, nothing to stop that being another cjobject!

.add , job

End If

Next job

End With

' now we can use the library object to driver dependencies

With infoJob.init(Nothing)

'the info file name .. we'll try to mirror the structur eof the google apps script/drive version

' preamble

.add "title", dependencyList.toString("name")

.add "committedDate", 0

.add "createdDate", getTimestampFromDate()

.add "modifiedDate", getTimestampFromDate()

.add "version", settings.toString("APP.VERSION")

.add "noticed", getTimestampFromDate()

.add "extract", settings.toString("EXTRACT.TO") & dependencyList.toString("name") & "/"

.add "fileName", settings.toString("FILES.INFO")

.add "fileId", .toString("extract") & .toString("fileName")

'module list

modulesToInfo dependencyList.child("scripts"), _

.add("modules").addArray, _

.toString("extract"), _

settings.toString("FOLDERS.SCRIPTS")

.add "extracted", False

.add "repo", dependencyList.toString("name")

'dependency list

modulesToInfo library, _

.add("dependencies").addArray, _

.toString("extract"), _

settings.toString("FOLDERS.DEPENDENCIES")

' add excel references

makeExcelReferences project.getObject("project"), .add("excelReferences").addArray

.add "readmeFileId", .toString("extract") & settings.toString("FILES.README")

.add "dependenciesFileId", .toString("extract") & settings.toString("FILES.DEPENDENCIES")

.add "crossFileId", .toString("extract") & settings.toString("FILES.CROSS")

End With

library.tearDown

Set makeInfoFile = infoJob

End Function

Private Function modulesToInfo(moduleJob As cJobject, infoJob As cJobject, _

extract As String, folderName As String) As cJobject

Dim job As cJobject, jo As cJobject, modl As cVBAmodule, fileName As String

With infoJob

For Each jo In moduleJob.children

' this is if we are using an indirection for the library

If jo.isObjValue() Then

Set job = jo.getObject

Else

Set job = jo

End If

' get the module

Set modl = job.getObject("module")

fileName = modl.name & _

conditionalAssignment(modl.textKind = "ClassModule", ".cls", ".vba")

With .add

.add "name", modl.name

.add "type", modl.textKind

.add "folder", concatFolderName(extract, folderName) & "/"

.add "id", concatFolderName(.toString("folder"), fileName)

.add "fileName", fileName

.add "docsName", modl.name & _

conditionalAssignment(modl.textKind = "ClassModule", "_cls", "_vba") & ".md"

.add "docsId", concatFolderName(.toString("folder"), .toString("docsName"))

End With

Next jo

End With

Set modulesToInfo = infoJob

End Function

Private Function mdWrap()

mdWrap = " " & vbLf

End Function

'-- these are about making the content for documentation files

'--------------------------------------------------------------

Private Function makeCross(cross As cJobject, info As cJobject) As String

Dim c As cStringChunker, job As cJobject

Set c = New cStringChunker

c.add("# VBA Project: ").addLine (info.toString("title"))

c.add("This cross reference list for repo (").add(info.toString("repo")).add(") was automatically created on ").add(CStr(Now())).add (" by VBAGit.")

c.addLine ("For more information see the [desktop liberation site](https://ramblings.mcpher.com/drive-sdk-and-github/getting-your-apps-scripts-to-github/ ""desktop liberation"")")

c.add ("You can see [library and dependency information here](")

c.add(getVGSettings().toString("FILES.DEPENDENCIES")).addLine(")").addLine ("")

c.addLine ("###Below is a cross reference showing which modules and procedures reference which others")

c.addLine "*module*|*proc*|*referenced by module*|*proc*"

c.addLine "---|---|---|---"

For Each job In cross.children

' the module being referenced

If (isModuleObj(job.getObject("proc"))) Then

c.add (job.getObject("proc").name)

Else

c.add (job.getObject("proc").parent.name)

End If

c.add ("|")

' the proc doing the referencing

If (Not isModuleObj(job.getObject("proc"))) Then

c.add (job.getObject("proc").name)

End If

c.add ("|")

' the module doing the referencing

If (isModuleObj(job.getObject("by"))) Then

c.add (job.getObject("by").name)

Else

c.add (job.getObject("by").parent.name)

End If

c.add ("|")

' the proc doing the referencing

If (Not isModuleObj(job.getObject("by"))) Then

c.add (job.getObject("by").name)

End If

c.addLine ("")

Next job

makeCross = c.toString

End Function

Private Function makeReadMe(info As cJobject) As String

Dim c As cStringChunker

Set c = New cStringChunker

c.add("# VBA Project: ").addLine (info.toString("title"))

c.add("This repo (").add(info.toString("repo")).add(") was automatically created on ").add(CStr(Now())).add (" by VBAGit.")

c.addLine ("For more information see the [desktop liberation site](https://ramblings.mcpher.com/integrate-vba-with-github/ ""desktop liberation"")")

c.add ("you can see [library and dependency information here](")

c.add(getVGSettings().toString("FILES.DEPENDENCIES")).addLine(")").addLine ("")

c.add ("To get started with VBA Git, you can either create a workbook with the [code on gitHub](https://github.com/brucemcpherson/VbaGit ""VbaGit repo"")")

c.add (", or use this premade [VbaBootStrap workbook](https://ramblings.mcpher.com/wp-content/uploads/2020/02/VbaGitBootStrap.xlsm ""VbaBootStrap"")")

c.add (mdWrap)

c.add ("Now update manually with details of this project - this skeleton file is committed only when there is no README.md in the repo.")

makeReadMe = c.toString

End Function

Private Function makeDependency(project As cJobject, info As cJobject) As String

Dim c As cStringChunker, job As cJobject, settings As cJobject, jo As cJobject

Set settings = getVGSettings(True)

Set c = New cStringChunker

c.add("# VBA Project: ").addLine (info.toString("title"))

c.add("This repo (").add(info.toString("repo")).add (") was automatically created on ")

c.add(CStr(Now())).add (" by VBAGit.")

c.add ("For more information see the [desktop liberation site](https://ramblings.mcpher.com/drive-sdk-and-github/getting-your-apps-scripts-to-github/ ""desktop liberation"")")

c.add (" or [contact me on G+](https://plus.google.com/+BruceMcpherson ""Bruce McPherson - GDE"")")

c.add (mdWrap)

c.add("## Details for VBA project ").addLine (info.toString("title"))

c.add ("Where possibile directly referenced or sub referenced library sources have been copied to this repository")

c.add (mdWrap)

c.add("### Modules of ").add(info.toString("title")).add(" included in this repo").addLine

c.add("*name*|*type*|*source*|*docs*").add(mdWrap).add("---|---|---|---").add (mdWrap)

For Each job In info.kids("modules")

c.add(job.toString("name")).add("|").add(job.toString("type")).add ("|")

c.add("[").add(job.toString("fileName")).add("](").add(settings.toString("FOLDERS.SCRIPTS")).add ("/")

c.add(job.toString("fileName")).add(" ""script source"")").add ("|")

c.add("[").add(job.toString("docsName")).add("](").add(settings.toString("FOLDERS.SCRIPTS")).add ("/")

c.add(job.toString("docsName")).add (" ""script docs"")")

c.add (mdWrap)

Next job

c.add(mdWrap).add("### All dependencies and sub dependencies in this repo").add (mdWrap)

c.add("*name*|*type*|*source*|*docs*").add(mdWrap).add("---|---|---|---").add (mdWrap)

For Each job In info.kids("dependencies")

c.add(job.toString("name")).add("|").add(job.toString("type")).add ("|")

c.add("[").add(job.toString("fileName")).add("](").add(settings.toString("FOLDERS.DEPENDENCIES")).add ("/")

c.add(job.toString("fileName")).add(" ""library source"")").add ("|")

c.add("[").add(job.toString("docsName")).add("](").add(settings.toString("FOLDERS.DEPENDENCIES")).add ("/")

c.add(job.toString("docsName")).add (" ""library docs"")")

c.add (mdWrap)

Next job

c.add(mdWrap).add("###Excel references").add (mdWrap)

If (info.child("excelReferences").children.Count > 0) Then

c.add ("####These references were detected in the workbook (")

c.add (project.getObject("project").wBook.name)

c.add (") this repo was created from. You may not need them all")

c.add (mdWrap)

' do the table titles

For Each job In info.child("excelReferences").kids(1)

c.add("*").add(job.key).add ("*|")

Next job

c.chopIf("|").add (mdWrap)

For Each job In info.child("excelReferences").kids(1)

c.add ("---|")

Next job

c.chopIf("|").add (mdWrap)

' now the content

For Each jo In info.kids("excelReferences")

For Each job In jo.children

c.add(job.cValue).add ("|")

Next job

c.chopIf("|").add (mdWrap)

Next jo

c.chopIf("|").add (mdWrap)

Else

c.add ("####No references were detected in the workbook (")

c.add (project.getObject("project").wBook.name)

c.add (") this repo was created from.")

c.add (mdWrap)

End If

c.add (mdWrap)

c.add ("You can see [full project info as json here](")

c.add(info.toString("fileName")).add (")")

makeDependency = c.toString

End Function

Private Function constructModLink(name As String, folder As String, fileName As String, hover As String)

Dim c As cStringChunker

Set c = New cStringChunker

c.add("[").add(name).add ("](")

c.add(folder).add ("/")

c.add(fileName).add (" """)

If hover <> vbNullString Then

c.add hover

Else

c.add name

End If

c.add("""").add (")")

constructModLink = c.toString

End Function

Private Function makeArguments(modl As cVBAmodule, info As cJobject) As String

' this will make a mardown string for all the procedures and arguments in this module

Dim c As cStringChunker, proc As cVBAProcedure, a As cVBAArgument

Set c = New cStringChunker

c.add("# VBA Project: **").add(info.toString("title")).addLine ("**")

c.add("## VBA Module: **").add(findModLink(modl.name, info, "source is here", "fileName")).addLine ("**")

c.add("### Type: ").add(modl.textKind).add(" ").addLines (2)

c.add("This procedure list for repo (").add (info.toString("repo"))

c.add(") was automatically created on ").add(CStr(Now())).addLine (" by VBAGit.")

c.addLine ("For more information see the [desktop liberation site](https://ramblings.mcpher.com/drive-sdk-and-github/getting-your-apps-scripts-to-github/ ""desktop liberation"")")

c.addLine.add("Below is a section for each procedure in ").add (modl.name)

For Each proc In modl.procedures

c.addLines(2).addLine ("---")

c.add("VBA Procedure: **").add(proc.name).add("**").add(" ").addLine

c.add("Type: **").add(proc.procTextKind).add("**").add(" ").addLine

c.add("Returns: **").add(findModLink(proc.procReturns, info, , "docsName")).add("**").add(" ").addLine

c.add("Return description: **").add(proc.returnDoc).add("**").add(" ").addLine

c.add("Scope: **").add(proc.scope).add("**").add(" ").addLine

c.add("Description: **").add(proc.description).add("**").add(" ").addLine

c.addLine.add("*").add(proc.declaration).add("*").add(" ").addLines (2)

If (proc.arguments.Count > 0) Then

c.addLine "*name*|*type*|*optional*|*default*|*description*"

c.addLine "---|---|---|---|---"

For Each a In proc.arguments

c.add(a.name).add ("|")

c.add (findModLink(a.argType, info, , "docsName"))

c.add("|").add(a.isOptional).add ("|")

c.add(a.default).add ("|")

c.addLine (a.description)

Next a

Else

c.addLine ("**no arguments required for this procedure**")

End If

Next proc

makeArguments = c.toString

End Function

Private Function findModLink(modlName As String, info As cJobject, Optional hover As String = vbNullString, _

Optional fn As String = "docsName") As String

Dim job As cJobject, settings As cJobject, c As cStringChunker

Set c = New cStringChunker

Set settings = getVGSettings(True)

If hover = vbNullString Then

hover = modlName

End If

For Each job In info.kids("modules")

If (job.toString("name") = modlName) Then

findModLink = constructModLink(modlName, "/" & settings.toString("FOLDERS.SCRIPTS"), job.toString(fn), hover)

Exit Function

End If

Next job

For Each job In info.kids("dependencies")

If (job.toString("name") = modlName) Then

findModLink = constructModLink(modlName, "/" & settings.toString("FOLDERS.DEPENDENCIES"), job.toString(fn), hover)

Exit Function

End If

Next job

findModLink = modlName

End Function

' these are all about handling credentials

'-----------------------------------------

Public Function getFromVbaGitRegistry(key) As String

Dim j As cJobject

Set j = getVGSettings().child("REGISTRY")

getFromVbaGitRegistry = GetSetting(j.toString("root"), j.toString("app"), key)

End Function

Public Function setVbaGitRegistry(key, value) As String

Dim j As cJobject

Set j = getVGSettings().child("REGISTRY")

SaveSetting j.toString("root"), j.toString("app"), key, value

End Function

Private Function getGitBasicCredentials()

getGitBasicCredentials = getFromVbaGitRegistry( _

getVGSettings().toString("REGISTRY.basic"))

End Function

Private Sub setGitBasicCredentials(user As String, pass As String)

setVbaGitRegistry getVGSettings() _

.toString("REGISTRY.basic"), Base64Encode(user & ":" & pass)

End Sub

Private Sub setGitClientCredentials(clientId As String, clientSecret As String)

setVbaGitRegistry getVGSettings() _

.toString("REGISTRY.client"), _

Base64Encode(clientId & ":" & clientSecret)

End Sub

Private Function getGitClientCredentials()

getGitClientCredentials = getFromVbaGitRegistry( _

getVGSettings().toString("REGISTRY.client"))

End Function

For help and more information join our community, follow the blog or follow me on Twitter . For how to do this for Google Apps Script, see Getting your apps scripts to Github