Everlaw API (1.alpha)

The Everlaw API is organized around REST. Our API has predictable resource-oriented URLS, returns JSON-encoded responses, and uses standard HTTP response codes, authentication and methods.

The Everlaw API uses API keys to authenticate requests. You can view and manage keys for your organization on the Organization Admin page.

Your API keys grant access to your Everlaw platform data - be sure to keep them secure! Do not share your secret API keys in publicly accessible areas such as GitHub, client-side code, open S3 buckets, etc.

You should provide your API key as a Bearer token when making an API request by setting the "Authorization" header:

Authorization: Bearer everlaw-api.XXXX.YYYYYYYY

You must provide your API key with every request, and all API requests must be made over HTTPS. All API requests and responses are encrypted via TLS.

All responses are in JSON format. Response entities are encapsulated in a JSON object along with any other top-level response content, e.g. pagination links (see the Pagination section below):

{
   "data": {
       "id": 1,
       "name": "Test Project"
       ...
   }
}

Some API endpoints fetch lists of objects (e.g. listing all users in an organization). These list API endpoints share a common structure, taking at least these two parameters: limit and after. The Everlaw API utilizes cursor-based pagination via the after parameter. This parameter takes an existing object ID value and returns objects starting after the given ID. The limit parameter specifies how many objects you would like to return in a given request.

Paginated responses will include a links entry in the top-level response that provides pre-built pagination links for fetching more results (if they exist). This links object will always include a next key which is a URL to fetch the next page of results OR null if this was the last page of results:

{
   "data": [{
       "id": 1,
       "name": "Test Project"
       ...
   }],
   "links": {
       "next": "https://...."
   }
}

The Everlaw API employs rate limiting to help limit the performance impact of API activity and maintain stability. There is a fixed rate limit of 25 requests per second enforced per authenticating user account. Request rates that exceed this limit may see error responses with status code 429. Additionally, there is a stricter rate limit enforced for failed authentication (invalid/missing API tokens).

A basic approach for handling rate limiting is to watch for 429 status codes and employ a retry mechanism. This retry mechanism should follow an exponential backoff schedule (with increasing delays between successive retries) to reduce request rates when necessary.

When using the Everlaw API it is important to consider users that are interacting through the Everlaw platform - actions you take through the API (particularly those that create or modify platform objects, such as uploads/datasets) can impact the platform experience for your reviewers. To help keep this experience user-friendly and performant, Everlaw imposes some limits on how the API can be used, including the rate limits discussed above as well as operation-specific limits. These limits include caps on the number of user-visible objects created via the API (for example, a limit on the number of datasets that can be created via the API), and will be called out on the specific operations below.

If you hit one of these limits, the operation will fail with response code 422 (unprocessable) and include an error message indicating the particular limit. These limits can be adjusted for specific databases or organizations as necessary to allow for continued API use - please contact Everlaw Support for more information or if you would like to adjust a particular limit.

The Everlaw API uses conventional HTTP response codes to indicate the success or failure of an API request. In general, codes in the 2xx range indicate success; codes in the 4xx range indicate an error with the request content (e.g. a missing parameter, or insufficient permissions); codes in the 5xx range indicate an error with our servers (e.g. scheduled maintenance).

Error responses will include a JSON body with a human-readable title indicating the cause of the error and the HTTP status code:

{
    "title": "You do not have permission to make this request",
    "status": 403
}

Users on Everlaw are typically members of one or more organizations. These organizations are a convenient way to group and manage user accounts as well as case data (databases and projects) owned by the organization. Each organization is associated with a list of members (some of whom may be "org admins") as well as a list of databases and their associated projects that the organization owns.

Organizations are administered by members that have been granted the "org admin" role, allowing them administrative control over membership, SAML/SSO configuration, access to organization-owned cases, management permissions over organization API keys, and other functionality. Organization-level API keys are also granted this "org admin" role. For more details, see Organization Administrator.

Matters on Everlaw are organized into top-level entities known as databases. Each database is a separate collection of documents owned and administered by a particular organization. Every database has one or more associated projects, which contain a subset (possibly all) of the documents in the database, but which have their own entirely separate review work. For more details, see Database and Project Permissions.

Permissions

There are a few important actions possible at the database level - in particular, uploading and deleting documents and administering (creating/modifying/deleting) projects associated with the database. These are managed via permissions on the database objects. For more details, see Database Administration and Permissions.

At the project level, permissions are managed via User Groups, which comprise a set of permissions and a list of members granted those permissions. Users can be members of one or more groups, and are granted the union of all the permissions of those groups. For more details, see User Groups and Project Permissions.

Organization admins are by default granted administrative access to the databases owned by that organization (as well as all associated projects). It is possible to disable this org admin access for specific databases as necessary to limit org admin access; in that case, org admins will be able to fetch basic summary information about the database/projects (e.g. its name, total billable size, etc) but not access documents or other similar data. This distinction is reflected in the separation of "Database/Project Org Admin" endpoints from others at the database and project levels. For more details, see Organization Admin: Organization and Project Administration.

Matters on Everlaw are billed based on the amount of data hosted each month. There are several endpoints available to fetch billing information for a particular month by database or project or across an entire organization. This billing information includes sizes of the hosted data by data type (native, pre-processed, produced) and storage type (standard, ECA) as well as project/database status information relevant for billing - e.g. when a project was suspended or deleted.

Monthly billing information will include both final sizes and peak sizes. Final sizes reflect the size at the end of the month (or the current date, if the month has not yet ended), whereas peak sizes reflect the maximum size observed over the given month (which is the basis for computing your actual bill). Each size point includes sizes for Standard and ECA (Early Case Assessment) data, which are further broken down by data type.

Monthly billing information also includes status information that reflects the state of the project/database over the course of the month. This includes the effective state of the project for the sake of billing (i.e. do we consider the project to have been active/suspended/deleted for the month), the final/end-of-month state for the project, and timestamps for important state changes (e.g. suspension, deletion).

When requesting billing information for a database, our response will include separate billable size and state information for each project in that database, as well as a combined overall billable size and state for the database. The combined database size typically reflects the size of the largest project in that database, and the status is the merger of the status of all the projects (e.g. if any project is active, the database as a whole is considered active; if all projects are suspended, the database is considered suspended, etc).

Global operations.

Organization-level administrative operations.

Database-level operations.

Database-level administrative operations. Accessible to org admins and parent org admins even when the database has Org Admin access disabled.

Native uploading refers to the ingestion of native-only, unprocessed (and un-numbered/Bates stamped) data into the Everlaw system. During ingestion, the Everlaw processing system will extract child documents (individual emails from an Outlook PST, compressed files from a ZIP, filesystem data from a forensic disc image, etc.), generate PDFs, and extract text and metadata. As documents are extracted and processed they will become available for review on the Everlaw platform.

Native uploads are organized into objects called Datasets, which provide a means for grouping related source files (e.g. all data in a particular collection or from a specific custodian) as well as defining a shared processing configuration. Each dataset then contains one or more source files uploaded by the user that contain the native data they would like to load into Everlaw. As each source file is uploaded, it will automatically be processed by Everlaw and the extracted documents (and their PDFs, text and metadata) will be available for search and review on the platform.

When you create a Dataset, you can also define the processing configuration you would like to use while processing documents uploaded to it. These configuration options are described below, as well as on our platform processing documentation.

Chat Segmentation

This setting controls how chat conversations will be extracted from native container formats into Everlaw chat documents. Each chat document represents a “segment” of a larger conversation. Supported segmentation options:

• SPLIT_25 (Small) - segments will contain a maximum of 25 messages
• SPLIT_50 (Medium) - segments will contain a maximum of 50 messages
• SPLIT_100 (Large) - segments will contain a maximum of 100 messages
• DAILY - each chat segment will contain messages sent in a single 24-hour period in the upload time zone

Deduplication

This setting configures how to handle exact duplicate documents encountered during native processing. Exact duplicates are determined based on a SHA1 hash of the native file (with custom hashing for certain filetypes, including emails - see our platform documentation for more details). There are three supported options:

• NONE - no documents in the dataset will be deduplicated
• ALL - any documents in the dataset that match an existing natively-uploaded document (in any dataset) will be deduplicated
• WITHIN_CUSTODIAN - any documents in the dataset that match an existing natively-uploaded document (in any dataset) with the same custodian will be deduplicated

When a document is deduplicated, its path and custodian will be added to the All Paths and All Custodians fields on the existing documents.

You can also configure whether or not to "deNIST" the dataset - remove documents whose hashes appear in the NIST NSRL reference hash lists of known software files.

PDF Generation

This setting determines which document types we will attempt to create PDFs for during processing. There are three supported options:

• DEFAULT - all documents except for types that do not convert well to PDF (primarily spreadsheet-like formats, e.g. Excel, CSV, etc) will be converted to PDF
• ALL - all documents (including Excels, CSV, etc) will be converted to PDF
• NONE - no PDFs will be generated

Image Inlining

This setting determines how and when images are inlined into email bodies (versus extracted as attachments) during processing. This helps prevent the proliferation of attached footer/signature images and icons being extracted as child documents. There are three supported options:

• SMART - all images that are explicitly inlined (included in the email body) or match Everlaw's heuristics for images that are intended to be inlined, are inlined - all other images are treated as attachments
• ALL - all images are inlined
• STRICT - only images that are explicitly inlined (included in the email body) are inlined - all other images are treated as attachments

OCR Language

This setting allows you to specify the particular language to detect when doing optical character recognition (OCR). By default, this is set to 'auto', which can detect and extract all Latin-alphabet languages (e.g. English, French, German) as well as Chinese, Japanese and Korean (CJK languages), with the caveat that autodetection will only detect and extract one language (in addition to English) per page.

You can alternatively provide an ISO/DIS 639-3 language code of a specific supported language you would like to detect (e.g. 'rus' for Russian) - in this mode, OCR will only detect that one language and English.

For more details about configuring an OCR language, see our platform processing documentation.

Page Size

This setting controls the default page size used when generating PDFs. There are two supported options:

• Letter
• A4

Hyperlinked Images

This setting controls whether hyperlinked images are fetched and displayed in the body of emails.

Slack Attachments

This setting controls whether attachments are fetched for Slack messages.

Slack exports chat attachment files as download links to files.slack.com. Attachment files can be fetched as long as the export token remains valid.

PowerPoint Speaker Notes

This setting controls whether PowerPoint speaker notes are included when generating PDF and text files. There are three supported options:

• INCLUDE - include speaker notes with original formatting. PowerPoint notes pages will be displayed in the PDF without modification.
• INCLUDE_STREAMLINED - include speaker notes with optimized formatting. Slides will be scaled to optimize legibility, with speaker notes text displayed under the slide. Any additional objects in the notes page, such as headers and footers, will be excluded from the PDF.
• EXCLUDE - speaker notes will be excluded from PDF and text files, and slides will be full-page in the PDF

Timezone

This configuration setting is used to specify the timezone from which the documents were originally collected. This timezone will be assumed for any extracted dates (e.g. metadata) that do not specify an explicit timezone. Email date headers will also be displayed in this timezone when generating PDFs. The value should be a timezone name from the tz (IANA time zone) database, e.g. "America/Los_Angeles", "Europe/London".

Partial Projects

This configuration allows you to specify which partial projects this upload should be included in. Documents uploaded to a database will always be included in all complete projects in that database. By default, they are not included in any partial projects.

Custodians

When creating a dataset, you can specify a default custodian to apply to every file that gets uploaded within it. Later, when uploading a source file to a dataset, you can specify a custodian to apply to that specific file and all its children which will take precedence over the default dataset custodian. You can also specify a mapping from child path (within that source file) to custodian for fine-grained control - for example, if you're uploading a ZIP file with the following directories:

• Jane Doe - Emails/
• Jane Doe - Hard Drive/
• Mike Smith - Emails/

You can specify a child custodians map of

{
    "Jane Doe - Emails/": "Jane Doe",
    "Jane Doe - Hard Drive/": "Jane Doe",
    "Mike Smith - Emails/": "Mike Smith"
}

The end result will be the following:

• Jane Doe - Emails/ - Custodian "Jane Doe"
• Jane Doe - Hard Drive/ - Custodian "Jane Doe"
• Mike Smith - Emails/ - Custodian "Mike Smith"

Passwords

Passwords can be supplied when uploading a new source file. These passwords will be used when attempting to open any encrypted file found in the entire dataset (including child documents and other source files).

There are currently two options for uploading new source files to Everlaw - multipart uploading and providing a direct link (URL). The first, multipart uploading, is applicable when you have the file locally (e.g. on a filesystem attached to the computer you're connecting to the Everlaw API). The second, providing a direct link, is applicable when the file you wish to upload is hosted on a publicly accessible server or cloud storage service (e.g. Amazon S3, Google Cloud Storage) to which Everlaw can connect directly.

Multipart Uploading

When sending local files to Everlaw using multipart uploading, each file is transferred in one or more separate parts that are uploaded independently. This allows for parallel, error-resilient and resumable uploading of very large files (up to 5TB). Each part is a contiguous section of the binary content of the file, and can be uploaded in any order. After initiating a source file upload and determining an appropriate part size, you can then upload each part of the file separately - for each part, you request a pre-authenticated URL from Everlaw to which to send the part's content (as a PUT request). After a part is successfully uploaded, the response will include a short string identifier called an ETag; after all parts are finished, you send the collection of ETags to Everlaw to assemble the parts together into the complete file and start processing.

For example, if you're trying to upload a 50GB file, you might select a 50MB part size, so that you have 1,000 parts: part number 1 contains byte range 0-50MB, part number 2 has 50MB-100MB, etc, ... For each part (and in parallel/any order), you request a pre-authenticated URL via the Everlaw API to which you PUT that particular byte range. If any of those individual part uploads fails (e.g. due to network issues), you can request a new URL for that same part and re-try that particular upload without affecting the other parts. Additionally, if the entire upload needs to be resumed, you can list all the parts that have been successfully created (as well as their ETags) via the Everlaw API, determine which have not yet been received, and re-upload only those parts without losing the progress you've already made. Finally, after collecting the 1,000 ETags from each of those uploads, you send those in part order in a final request via the Everlaw API which triggers the assembly of those parts into your original 50GB file and begins processing.

There is a maximum of 10,000 parts per upload and each part must be 5MB to 5GB with the exception of the final part which has no minimum size. For example, if your file is 50.1MB bytes long, possible options include uploading a single 50.1MB part or uploading 5 parts of 10MB plus a final part of 0.1MB. For smaller files (< 5GB) it is reasonable to upload the entire file as a single part, but the 5GB part size limit means that any larger files must be split into multiple parts. A reasonable part size algorithm is:

def determine_part_size(total_file_size):
    # Maximum of (50MB, 1/10000 of the total file size rounding up)
    return max(50000000, 1 + total_file_size / 10000)

For more details about multipart uploads, see the endpoint-specific documentation below.

Direct Links

If the file you are uploading is hosted on a publicly accessible server or cloud storage service, you can provide a direct link (URL) to which Everlaw can connect directly to download the file. These links must not require any additional authentication (API keys, username/password credentials, etc.) other than what is included in the link itself. The server host must also support Range requests via the Range header. Examples of suitable links include pre-signed Amazon S3 URLs and Google Cloud Storage URLs.

Direct links can be provided when creating a new source file. If a link is provided, Everlaw will immediately begin downloading the file and will begin processing when that download finishes; no additional client action is necessary.

To demonstrate the overall upload process, let's assume we have a 65GB password-protected ZIP we wish to upload. First, we need to determine what dataset we'd like to use - we may already have an existing dataset with our desired processing configuration that we want to add this file to, or we may want to create a new dataset. Let's create a new dataset called "Upload Walkthrough" in our database (id 555) with our desired configuration using the PostDatabaseDataset operation:

POST /v1/databases/555/datasets

{
   "name": "Upload Walkthrough",
   "description": "A demo dataset for walking through the upload process",
   "deduplication": "NONE",
   "timezone": "America/Los_Angeles"
}

Response:

{
  "data": {
      "id": 1201,
      "name": "Upload Walkthrough",
      ...
  }
}

With our target dataset (id 1201), we can create a new source file using the PostDatasetFile operation. We have the file on our local filesystem (not publicly accessible), so we can't provide a directLink; we also want to include password and custodian information. Assuming the ZIP has a few different subdirectories for different custodians, we can set those using the childCustodians parameter:

POST /v1/databases/555/datasets/1201/sourceFiles

{
   "filename": "my_encrypted_collection.zip",
   "passwords": ["my_zip_password"],
   "childCustodians": {
       "Custodian_A_Collection/": "Custodian A",
       "Custodian_B_Collection/": "Custodian B"
   }
}

Response:

{
  "data": {
      "id": 988,
      "state": "UPLOADING",
      ...
  }
}

The source file (id 988) is now in state "UPLOADING" and we can start uploading the individual parts - let's choose a part size of 100MB, giving us 650 parts. We then upload each of those parts - let's walk through part number 25. First, we request a pre-authenticated URL for that part using the PostSourceFilePart operation:

POST /v1/databases/555/sourceFiles/988/parts/25

Response:

{
  "data": {
      "partNumber": 25,
      "url": "https://everlaw-demo.s3.amazonaws.com/..",
      ...
  }
}

Next, we PUT the specific byte range for this part to the generated URL and save the resulting ETag:

PUT https://everlaw-demo.s3.amazonaws.com/...

(bytes 24,000,000,001 - 25,000,000,000, 1-indexed, inclusive)

Response:

HTTP/1.1 200
ETag: "4bb64c06b1a72539e6d3476891daf17b"
...

We then save this ETag header for part 25; once we have all 650 ETags, we can complete the upload (optionally including a SHA1 hash of the entire file contents) using the PostSourceFile operation:

POST /v1/databases/555/sourceFiles/988

{
  "eTags": [..., "4bb64c06b1a72539e6d3476891daf17b", ...]
  "sha1Hash": "0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33"
}

Response:

{
  "data": {
      "id": 988,
      "state": "PROCESSING",
      ...
  }
}

The source file upload is now complete and the ZIP is processing - the contents will soon be available for review on the platform as Everlaw documents!