Agree the test conditions before opening the demo
Choose the platforms, account types and content formats that matter to your organization. Ask each vendor to state supported behavior and normal capture timing for those combinations. Agree the acceptance conditions before running the test, using the requirements checklist if you need a starting point.
Use an approved test account and harmless test content. Get permission for publication, edits, deletion and connection changes. Do not delete real public records or interrupt a production account just to demonstrate a feature. Where an action cannot be performed safely, request a separate demonstration and record the capability as unverified.
Keep a source log: platform, account, source identifier, action time, time zone and expected result. Assign a unique phrase such as “Archive evaluation Cedar 14” so the test items can be found without mixing them with unrelated records. This phrase is fictional test data.
Test the record from capture to production
Repeat relevant checks for each platform and account type in scope. A pass on one platform does not establish coverage on another. Wait the agreed observation interval before judging a result, and record measured delays rather than assuming capture is immediate.
| Check | What to do | What to inspect |
|---|---|---|
| T01 · Account connection | Connect an approved test account with the documented permissions. Confirm its identity and connection status. | The correct account is connected and permission limits are recorded. |
| T02 · Post and media capture | Publish a harmless uniquely named test post with an image. Record its source ID and publication time. Look for it after the agreed capture interval. | Supported text, media and source details match; actual capture delay is recorded. |
| T03 · Conversation context | Add a permitted test comment and reply. Find both in the archive and open the parent post. | Supported comments and replies remain associated with the correct conversation. |
| T04 · Edit history | After verifying initial capture, edit the test post on a platform that supports edits. Record the change time and inspect after the agreed interval. | The captured original and changed version are distinguishable, with available timing and metadata. |
| T05 · Source deletion | After verifying capture, delete only the approved test item at source. Record the time and inspect the archive after the agreed interval. | The captured item remains retrievable and deletion behavior matches the written coverage statement. |
| T06 · Search accuracy | Use the unique test phrase, account and date range. Include a date boundary and check the time zone. Compare returned items with your test inventory. | Expected items can be found; unexpected omissions or inclusions are explained. |
| T07 · Export usability | Export the test conversation. Open the package outside the archive and inspect text, media, metadata and version relationships. | A colleague can read the package without a vendor login; missing elements and size limits are documented. |
| T08 · Integrity verification | Where a checksum is supplied, verify the export using the vendor instructions. Change a COPY and repeat; preserve the original. | The original verifies and the changed copy does not; the method and what it proves are understood. |
| T09 · User permissions | Sign in as an approved reviewer with limited access. Try agreed search, export and administrative actions. | Allowed actions work and restricted actions are denied for that role. |
| T10 · Connection interruption | In an isolated test environment, arrange an approved connection interruption and reconnect. If unavailable, request a demonstration and record it as unverified. | Alerting, recovery steps and any unrecoverable interval match the written explanation. |
| T11 · Historical availability | Identify a known pre-connection item and ask whether it can be retrieved. Record account, platform, date and permission limits. | Available history and excluded history are documented; backfill is not confused with archive import. |
| T12 · Request rehearsal | Give a second reviewer a fictional request for the test phrase and date range. Time search, review and export separately. | The reviewer produces the expected records and records manual steps, gaps and total effort. |
Test two different questions separately. Deleting an item after you have confirmed capture tests preservation. Deleting it before confirmed capture tests exposure to timing gaps. If you run the second exercise, agree it with the vendor and record the timing; do not treat the two results as equivalent.
Give the export to someone who did not run the demo
A guided demonstration can hide the work a new reviewer must do. Give a colleague a short fictional request: “Find the posts and related test comments containing Archive evaluation Cedar 14 during the agreed test window, including captured versions and deletions.”
Have that colleague run the search, inspect the thread and produce an export. Time those stages separately. Ask them to open the package without the archive application and explain what it contains. If they need vendor help, conversion software or a manual index, record that effort.
Check the start and end of the date range, especially if the interface and export use different time zones. Reconcile results against your source log. Treat missing items as questions to investigate: an unsupported content type, a permissions issue and a search mistake need different remedies.
This exercise tests retrieval and production. Your normal review, redaction and disclosure approval process still determines what may be released.
Use results that preserve uncertainty
| Result | Use it when |
|---|---|
| Pass | You observed the agreed outcome and kept evidence. |
| Conditional | The outcome depends on a limitation or workaround that needs acceptance. |
| Fail | The observed result does not meet the agreed requirement. |
| Not tested | You have not directly verified the capability. This is the default. |
| Not applicable | The test is outside the agreed scope; record why and who agreed. |
The worksheet records a result, evidence reference, observed delay, issue owner and retest date for each check. Duplicate it for each vendor and platform combination. Keep the same source log and pass conditions so results remain comparable.
Review mandatory checks before comparing convenience or speed. A result of “conditional” is not a pass until the responsible owner accepts the limitation. Leave untested capabilities visible. The worksheet deliberately avoids an overall percentage that could hide an important failure.
Write down what the pilot cannot establish
A successful small test does not prove behavior during every API outage, every volume spike or every source-platform change. Ask how changes to coverage are communicated, how connection problems are detected and what recovery can retrieve after an interruption.
Capture the software version or demonstration date, the account permissions and any vendor assistance. Keep the original exports with their integrity evidence and your observation log. Record security and contractual questions separately; a functional pilot is only one part of the purchase decision.
End with named reviewers and a decision: proceed, proceed subject to explicit conditions, extend the evaluation, or do not proceed. Retest failed or conditional mandatory checks after a fix. If you are replacing a provider, also prove the historical access and transition plan.
Get the PDF workbook
The evaluation scorecard includes: test conditions, a source log, twelve step-by-step checks, result selectors, evidence and retest fields, and a reviewer decision. Use a separate copy for each vendor and platform combination.
Complete the form and your download will appear here.
The form could not load. Try again, or check whether your browser is blocking the form.
Still having trouble? Contact Brolly.
Brolly will use your details to respond to your request. Read our privacy policy.
Your workbook is ready
Download the PDF and open it in a PDF reader to fill in the shaded fields, or print a copy. Save your completed file before closing. For expandable fields and extra rows, use the browser worksheet and choose Save my copy. Worksheet answers stay on your device and are not sent to Brolly.
Common questions
How long should a pilot take?
Use enough time to run the agreed checks, investigate exceptions and involve your reviewers. A short demonstration can introduce the workflow, but an evaluation may need multiple capture cycles and retests. Set the schedule around your purchasing and renewal deadlines.
Can we test using real official accounts?
Only with your organization’s permission and an agreed plan. Use isolated test content for edit and deletion exercises. Do not interrupt production capture or remove real records as part of an unapproved test.
Does a verified checksum prove the archive captured everything?
No. A checksum helps detect changes to the file being verified. Capture coverage, source accuracy and export integrity are different questions and need separate evidence.
What if a platform does not support the test action?
Document the limitation and decide whether the capability is required. Use Not applicable only when the requirement is genuinely outside your agreed scope; use Fail or Conditional when an in-scope requirement is unmet.
Run the test with Brolly
Bring your test account scope and acceptance conditions. We can walk through the relevant capture, search and export workflow and discuss the limits that apply to your platforms.