What is Zamba Cloud?

Zamba Cloud makes it easier to handle large amounts of camera trap data for research and conservation.

Zamba Cloud uses machine learning to automatically detect and classify animals in camera trap videos or images. You can use Zamba Cloud to:

• Classify which species appear in an image or video, including identifying footage without animals
• Train a custom model with your own labeled data to identify species in your habitat
• And more 🙈 🙉 🙊

All without writing a line of code!

To generate predictions for unlabeled videos/images

Zamba includes multiple state-of-the-art, pretrained machine learning models for different contexts. Each is publicly available and can be used to classify new videos/images.

To generate new predictions, you'll upload media files and receive a spreadsheet telling you what species are most likely present in each, allowing you to weed out false triggers and get straight to the files of interest.

After creating an account, you can either upload media files directly or point Zamba Cloud to an FTP server where your files are stored. Then, get back to your day while Zamba Cloud processes your media. You'll get an email when it's done. Simply log back into your account and a spreadsheet with labels for each media file will be waiting for you!

Step by step instructions

To train a model based on labeled videos/images

Users can upload additional labeled videos/images, and Zamba will train a new custom model.

After creating an account, you'll upload a set of videos/images and their correct species labels. These will be used to improve the base classification model (African species model for videos, the Global species model for images), building on what the model has already learned from thousands of training examples. The model can even learn to predict entirely new species and new ecologies, no matter where they are! Then you can go do some birdwatching, and you'll receive an email when your new model is ready. Your specialized model will then be available to classify any new media you upload to Zamba Cloud!

What models are available in Zamba Cloud?

Zamba Cloud comes with a few official model options that are pretrained for different contexts. The table below summarizes their relative strengths.

All models except the African species (slowfast) model use an image-based architecture. The slowfast model uses a video-native architecture, which in some cases can better capture motion to detect small species. Check out the zamba python docs for more model details.

How accurate is Zamba Cloud?

To assess accuracy, model predictions were compared with labels manually applied by researchers and citizen scientists. Performance was measured on data that were not part of the model's training set, which provides a more accurate picture of how the model will perform on other new data (like yours!).

Species classification for video

• Top-1 accuracy is how frequently the correct label is the top label predicted by the model, e.g., the African species video model predicted the correct species for 82% of videos.
• Zamba Cloud outputs the top 3 predicted labels to enable researchers to easily surface videos/images of interest. Top-3 accuracy is how frequently the correct label is in the top three species with the highest predicted probability. Eg. For the African species model, the correct label was in the top three predicted for 94% of videos/pictures.

Species classification for images

It's harder to report metrics for our large, global model that will be meaningful for all use cases. For most relatively common taxa, F1 scores (combination of precision and recall) are between 0.85 and 0.99, indicating reliable performance for many applications. For your use case, you can threshold the per-species probabilities to capture more images (and more false positives for a species) or fewer images (fewer false positives, but some true positives may be missing).

We've observed some general limitations to this model. Generally, small species (rodents and reptiles) perform worse, and there are some known issues with commonly confused marsupials, some bird species, and different kinds of antelopes.

If you want to see better accuracy for your use specific case, we created this model to be a base for custom, user-created models for specific habitats and geographies, so train away!

Blank detection

All the species models contain a blank output label in addition to the species labels, but blank detection works slightly different between the image and video models. In the image model, blank means that no animal was detected by MegaDetector, which has its own accuracy docs. For videos, there is an additional "blank vs. non-blank" detection model. This means that for videos either a species model or the blank model can be used for blank detection. The key difference is that the blank model only outputs the probability that the video does not contain an animal and does not do any species classification for non-blank videos. The table below compares blank performance between the African species model and the blank model.

• Precision is the percent of videos/images that the model labels as blank that are actually blank. For example, 83% of the videos/pictures that the African species model labels as blank are actually blank.
• Recall is the percent of all blank videos/images that are correctly classified by the model. For example, the blank model correctly detects 89% of actually blank videos/pictures.

Our goal is to continually improve these algorithms, and you can help! The most valuable contribution to this effort is additional labeled data. Find out more about how you can submit a correction for the videos/images that Zamba Cloud got wrong. Or, if you have videos/images that are already labeled, you can share labeled data directly with us.

What species can Zamba Cloud identify?

Blank vs. non-blank model

The blank vs. non-blank model only outputs the probability that the video is blank, meaning that it does not contain an animal. It does not provide any species classification.

African species models

European species model

Global species (images) model

The global image model predicts on 178 classes at varying taxonomic levels. While this model is a good first start for off-the-shelf, we created this model to be a base that is used for custom, user-created models for specific habitats and geographies.

Don't see what you're looking for? We are always open to expanding the list of species that Zamba can identify—consider training your own model with your own data to add species.

Sign up or log in

To begin, either sign up to create an account or log in to your existing account.

How to classify unlabeled videos/images

This section walks through how to use Zamba Cloud if you don't know what species your footage contains and want to generate labels.

Tutorial

After logging in, you'll be taken to the Uploads page, where you will upload the media files you want Zamba Cloud to process.

There are two ways of submitting media files: Direct upload or FTP upload. Currently, the FTP upload option only supports videos.

Zamba supports dozens of image and video formats, so yours is likely covered. The specific list of video file formats is here, and for images, Zamba supports the file formats listed here.

Wondering how long it will take? See this FAQ question for more information.

Direct upload

Use this option to upload videos/images from local files on your computer.

1. Go to the Direct Uploads tab of the Uploads page. Click the + New Direct Upload button. Direct uploads work best with a fast internet connection, as limited bandwidth can cause the browser to time-out partway through the upload. If you have a slow internet connection, either check the fast upload box (which uses your computer's resources to reduce the size of the video before uploading), or consider an FTP upload instead.
2. Select what you want to upload: images or videos.
3. Drag and drop videos/images or click Upload Video(s)/Images(s) to select the media files you want to upload from your computer. When you are finished selecting, click Upload.
4. You may add an optional title and/or description. Use the ML MODEL dropdown menu to choose which of Zamba Cloud's pretrained models you'd like to use to classify your videos/images (if you have trained any of your own models, they'll appear here too!).
5. Click Begin Processing.

FTP upload

This option currently only works for videos. In order to use FTP upload, your videos have to already be uploaded to an FTP server. Many large organizations and universities run FTP servers, which can be used to host data for Zamba Cloud.

1. Go to the FTP Uploads tab of the Uploads page. Click the + New FTP Upload button.
2. Enter the FTP server URL path, username, and password in the corresponding fields and click Upload from FTP. Keep in mind that with the FTP submission, all the videos/images in the specified folder will submitted to Zamba Cloud. You will not be able to pick and choose which videos/pictures within the folder are processed.
3. Use the ML MODEL dropdown menu to choose which of Zamba Cloud's pretrained models you'd like to use to classify your videos/images (if you have trained any of your own models, they'll appear here too!).
4. Click Begin Processing.

Downloading species labels

Once you have submitted data files, you can see their status on the Uploads page under the corresponding "Direct Uploads" or "FTP Uploads" tab. The status will say "Zamba processing succeeded" when the label spreadsheet is ready for download.

Since it can take a few days to process a large quantity of files, we'll send you an email from zamba@drivendata.org when the labels are ready. In the meantime, feel free to close the webpage and take a walk.

The link in the email will take you to your Uploads page, where there will be a button that says Download Labels. Click on this to download the csv file, which can be opened in Excel, Numbers, Google Sheets, or read by analytic software like R or Python.

You can re-download the labels at any point from your Uploads page when you are logged in. Once your media files are uploaded, you can also generate labels with other models by clicking Run different model.

Understanding the labels spreadsheet

For each species label in the list above, Zamba Cloud uses an advanced computer vision model to estimate the probability that that label applies to the video or image. Probabilities range between 0 and 1. For each species label, 0 means the species is definitely not in the video, 0.5 means there's a 50% chance the species is in the footage, and 1 means the species is definitely in the video or image. For example, if the column for blank probability is 0.95 there is a 95% chance that the media has no animals in it.

The label spreadsheet format differs slightly for videos and images, so we will describe them separately below.

Video labels

The labels spreadsheet has a row for each video. The columns are:

• media_uuid: a Zamba-generated unique ID for the video
• original_filename: original video filename (for direct uploads) or filepath (for FTP uploads)
• top_1_label through top_3_probability: The next six columns contain the name and corresponding probability for the top three "most likely" species (i.e. the labels with probabilities closest to 1). These columns are intended as a shortcut to make filtering for videos/images of interest easy.
• The remaining columns contain the probabilities for all of the possible labels, based on the model used for classification. These will not sum to 1 as there can be more than one species present in each video. Zamba Cloud individually estimates the probability that each species is present.

A labels spreadsheet could look like this:

Image labels

The labels spreadsheet has a row for each image. The columns are:

• filepath: a filepath with the filename matching the filename of the original upload
• detection_category: an integer representing the category of the object MegaDetector detected. This should often be 1 which represents "animal".
• detection_conf: the confidence MegaDetector reports for its object classification on a scale from 0.0 to 1.0
• x1: the horizontal coordinate of the top-left corner of the bounding box, with top left as the origin
• y1: the vertical coordinate of the top-left corner of the bounding box
• x2: the horizontal coordinate of the bottom-right corner of the bounding box
• y2: the vertical coordinate of the bottom-right corner of the bounding box
• The remaining columns contain the probabilites for all of the possible labels, based on the model used for classification.

Filtering videos/images by species

You can filter all of your uploaded files on the Review Predictions page. Adjust the threshold slider based on what level of certainty you'd like to have that the videos/images contain the given animal.

For more fine-grained filtering, use the labels spreadsheet downloaded in the previous step. A few basic approaches:

• Filter by the most likely species columns: top_1_label, top_2_label, top_3_label. The spreadsheet makes it easy to get a list of videos/images where you animal of interest is in the top 1, 2, or 3 most likely labels.
• Filter by a the probability column for a specific species. For example, say you want to see only files that have at least an 80% chance of containing a lion. Set a filter on the lion column for rows with a value of 0.8 or greater.

Either of these approaches can be used to filter out media files that contain no wildlife. Either filter for rows where none of the top 3 labels are blank, or apply a "less than" filter on the blank probability column.

Submitting corrections

Zamba Cloud relies on user-labeled data to improve its predictions. If you have videos or images where Zamba Cloud did not predict the right species, let us know!

The easiest way to submit corrections is using the downloaded labels spreadsheet. Right now this process is only supported for videos.

1. Fill in the corrected_label column - the last column in the spreadsheet.

   • The label you put in this column must exactly match the column name for the species. To see the correct name formatting, find the corresponding probability column for the correct species. Eg. chimpanzee_bonobo, domestic_cat
   • The file should be saved as a CSV, which should be an export option from your spreadsheet tool.
   • Columns other than media_uuid and corrected_label are ignored for the corrections spreadsheet so you can leave these exactly how they were downloaded from Zamba Cloud.
   • If you have multiple species in a video, copy the entire row so the same video appears twice with one corrected_label per row (as shown above for video 123-456).
   • For videos/images where the top_1_label is correct, the corrected_label column can be left blank or you can confirm the correct label by entering it in that column.
   
   

   Example spreadsheet of corrections:

   media_uuid	...	...	corrected_label
   9ab-c65	...	...	blank
   89a-000	...	...		← video where label is already correct
   123-456	...	...	duiker	← video with multiple species
   123-456	...	...	forest_buffalo

2. Select the Review Predictions tab and then the Submit batch corrections button in the upper right corner. Then drag and drop or click Upload File to select the corrections spreadsheet you want to upload.

3. Finally, click Submit correction.

How to train your own model

This section walks through how to train a model tailored to your own species or ecosystems. To train a model, you need to have a set of videos or images that are already labeled with the correct species. Your species can be either a subset of the ones that official models are already trained to predict, or completely new ones!

The more training data you have, the better the resulting model will be. We recommend having a minimum of 100 videos or images per species. Having an imbalanced dataset - for example, where most of the videos are blank - is okay as long as there are enough examples of each individual species.

Media labels

Save a csv file with the correct labels for each training file on your local computer. The labels file should have columns for:

• filepath: the name of the video or image. Media file names must be unique, otherwise the species labels will not be able to match the correct file.
• label: the correct species label. If your labels are a subset of the ones predicted by an official model, these should exactly match the pretrained model label as much as possible. If more than one species appears in a file, enter multiple rows with the same filepath value and a different label value.

Example label file:

Tutorial with direct uploads

Use this option to upload labeled training videos from local files on your computer.

Direct uploads work best with a fast internet connection, as limited bandwidth can cause the browser to time out partway through the upload. If you have a slow internet connection, either check the "fast upload" box (which uses your computer's resources to reduce the size of the files before uploading), or consider an FTP upload instead.

Tutorial using an FTP server

FTP server uploading is only supported for video files at the moment. In order to use this option, your training videos have to already be uploaded to an FTP server. Many large organizations and universities run FTP servers, which can be used to host data for Zamba Cloud.

Step by step instructions

1. From the ML Models tab, click either Train Model From Direct Uploads or Train Model From FTP Files depending on where your files are stored.

2. Upload your labeled media files.

   • For direct uploads: Select your media type — image or video — then drag and drop or click Upload Image(s)/Video(s) to select the media files you want to upload.
   • For FTP uploads: Enter the FTP server URL path, username, and password in the corresponding fields. Keep in mind that with the FTP submission, all the videos in the specified folder will submitted to Zamba Cloud. You will not be able to pick and choose which videos within the folder are processed.
3. Drag and drop or click Upload species label file to upload the CSV of your media labels.
4. Click Upload or Upload from FTP at the bottom. This step may take awhile if you are using a direct upload.
5. On the next screen, use the dropdown menus under Zamba Cloud Match to match the labels in your files to labels that the official models are already trained to predict. If your label is new, select None (use your label) from the dropdown. Then click Next: Summary.
6. Enter a display name and (optionally) a description. You can also double check your training media by clicking See all individual media file names and labels.
7. You will be asked if you want to make the trained model available to other users. This will mean that other Zamba Cloud users will be able to select this model when they run their own jobs; they won't have access to the files or labels you upload.
8. You will be asked if your media can be used for training other models. If you check this box, it indicates that the administrators of the site may use these media in the future to make improvements to the open-source algorithms that underlie Zamba Cloud. This does not make the media publicly available or available to other Zamba Cloud users.
9. Click Start training model. Your work is done! Models may take some time to train, so feel free to close the browser and go about your day.
10. You will get an email from zamba@drivendata.org when your model is ready. You can check the status in the ML Models tab. It will also be added as an option in the dropdown when you are choosing which model to use to classify new files.

FAQ

How does Zamba Cloud work?

How much does Zamba Cloud cost?

Currently, Zamba Cloud is supported pro bono by DrivenData. Please reach out about funding opportunities for fiscal sustainability. More powerful paid versions for heavy users may be available in the future.

Because your labels can always be downloaded as a spreadsheet, you are not locked in to continuing to use this tool to access the labels that the algorithm predicted.

How long will it take to process my videos?

Depending on network conditions, you can expect processing to take about 12 hours for 1,000 videos. If you're experiencing issues, try limiting uploads to fewer than 2,500 videos at a time. Keep in mind that your job may take longer if it is in the queue behind another user's long running job.

How does Zamba Cloud train new models from user data?

Computer vision algorithms are only as good as the labeled data they're trained on. As a result, our official models can only predict the species that are included in the labeled training data we had available. Luckily, Zamba Cloud is a quick learner!

If you are able to gather labeled images or videos of your species of interest, you can pass those to Zamba Cloud and train a new model to identify your species. We recommend having at least 100 labeled images or videos of a species to retrain a model.

The retraining process starts with one of the official models, which has already learned from thousands of hours of camera trap footage. The model then continues training on the new data that you've provided.

For example, say that you want to identify ostriches specifically (who wouldn't? They're huge!). Our official model can identify "Large flightless bird," but that also includes things like emus. If you provide new labeled media files of "ostriches" specifically, the model will be able to combine its existing knowledge of how to identify all large flightless birds with the new information about how to distinguish ostriches from other species.

How can I contribute to Zamba Cloud?

We're always looking for partners who can share their data to help us improve the accuracy on current species detection as well as expand to new species.

You can let us know where Zamba Cloud got things wrong by submitting corrected labels. If you have additional data that is already labeled, we'd love to hear from you at zamba@drivendata.org.

Can I access the algorithms behind Zamba Cloud?

The algorithms behind Zamba Cloud are openly available for anyone to learn from and use. You can find the latest version of the project codebase on GitHub.

This project is structured as an open source command line tool and Python package, where you can run inference, train your own models, and even make your models available to the community.

Where did the labeled media used in Zamba Cloud come from?