Using the DVUploader tool

Adapted from:

https://github.com/GlobalDataverseCommunityConsortium/dataverse-uploader/wiki/DVUploader,-a-Command-line-Bulk-Uploader-for-Dataverse

13/02/2025

When to use the DVUploader tool

The DVuploader tool can be a good alternative for uploading the files via the User Interface in the following cases:

• If you want to upload a lot of files;
• if you want to upload files with a complex hierarchical folder structure;
• when new files are being generated/added to a directory and Dataverse needs to be updated with just the new files;
• uploading of files needs to be automated, e.g. added to an instrument or analysis script or program.

Preparations

1. You will need to install the application first. You will need Java version 8 or higher to be able to install the DVUploader. See this link to download Java:
   https://java.com/en/download/.
   
   
2. Then, click this link to download the DVUploader: https://github.com/GlobalDataverseCommunityConsortium/dataverse-uploader/releases/download/v1.2.1/DVUploader-v1.2.1.jar. (Clicking this link results in a direct download.) This is version 1.2.1. You can check the link on top of this article to check if this is still the most recent version.
   
   
3. There are two option for placing this jar file:
   • If you are not very familiar with using the command line, the following steps are recommended:
     Place the files you would like to upload in you home directory, together with the .jar file.
     For example, in a folder similar to  /Users/Laurahuisintveld.
   • If you know how to change your working directory via the Terminal/Command prompt, you can store the .jar file into the directory containing a subdirectory with the files intended for upload.
     
     
4.  Find out the DOI of the dataset you would like to add files to. If there is no dataset yet, create one first.
   You will need the DOI in this format: doi:10.34894/ADBKTN
   
   
5. You will need you DataverseNL API-key. You can create and find your API-Token in the User Interface only. Login into DataverseNL, click on your account name in the top navigation bar and choose 'API Token' from the drop-down list. Note: this token will automatically become invalid after a year.
   
   

Terminal (Mac)/Command Prompt (Windows)

Open the Command Prompt (for Windows) or Terminal (MacOS). This program is standard available on your Windows/Mac computer. For Windows, type in 'cmd' in your search box. For MacOs, Search for 'Terminal' in the 'Spotlight Search' (top navigation bar) or in the Launchpad.

If you have placed your .jar file not in your 'home' directory, change the directory first in your Terminal/Command Prompt.

You need to adjust this line to suit your needs:

    java -jar DVUploader-v1.2.1.jar -key=<api key> -did=<dataset doi> -server=<server URL> -uploadviaserver -recurse <dir or file names> 

1. Replace <api key> with your DataverseNL API Token
2. Replace <dataset doi> with the DOI of your dataset.
3. Replace <server URL> with https://dataverse.nl (or use https://demo.dataverse.nl to try it out on the demo environment first.)
4. Replace <dir or file names> with the name of your folder you would like to upload, or paste a list of files names instead. The name of the folder can not contain a comma. If you are listing single files, also include the file extension.
5. If you don't want to keep the folder structure, remove '-recurse', from this line.

Example:

java -jar DVUploader-v1.2.1.jar -key=dc00d47f-5a49-4a82-93c6-458763a32a8e -did=doi:10.80227/PDVNL/KHN6P4 -server=https://demo.dataverse.nl -uploadviaserver -recurse Testdocumenten 

(This API-Token has been made invalid.) In the example above we are uploading the folder 'testdocumenten' with all its subdirectories to the dataset with doi 10.80227/PDVNL/KHN6P4. 

    6. Paste your adjusted line into the Command Prompt/Terminal.

The DVuploader will check if all the required values have been entered, and if so, the upload process will start. It checks for each file whether it already exists in the dataset, and if not, the upload is carried out. As soon as the DVUploader is completely ready, you will see:

***Execution Complete.***

A log file will be automatically created and stored in the same location as the .jar file. You can check the log (search for 'error') to see if any errors occurred. 

For MacOs Users: Sometimes hidden files with the .DS_Store extension are uploaded as well if files were compressed. Check after the upload if your datasets contain any and remove them before publishing.