A public benefit corporation founded in 2016

last updated 2021-02-11

{^Upstream Tech} is a public benefit corporation founded in 2016 by Marshall and Alden. It currently provides two services:

  1. {^Lens}, monitoring software for environmental organizations.
  2. {^HydroForecast}, a hydrologic forecasting service used by renewable energy utilities and conservation organizations.
In it's early days, we did a number of consulting projects, including two citizen science applications: the New York State's Herp Shark Count for the Galapagos Islands. After attending a lecture by Dr. Grey Nearing, who later became an advisor and briefly our Director of Research, we honed in a satellites and machine learning as tools we could leverage given our previous experience and technical strengths.

The first non-consulting project was a remote compliance tool for water transfers in the Western U.S. In the West, water rights are purchased or leased by conservation organizations with the aim to transfer water from agriculture use to environmental use to increase flows in streams and rivers for the benefit of freshwater species. Our service allowed organizations to remotely monitor that this transfer indeed occurred and that the water was not used on the farm field - a monitoring process that would typically require costly in-person visits througout the season. These visits diverted resources that could otherwise be applied to additional conservation projects. A classifer was used to predict the liklihood that a field was irrigated or not. The bones of this application later became our data processing pipeline for satellite imagery and machine learning.

An Upstream Tech prototype

From there, we became fascinated with anachronous design and complexity of water rights. We built a system that digitized a number of states' water rights, and used the same classifier as before to predict year-over-year water use. An aim was to build a database of water rights that on paper appeared low-value, but reliably applied water to their fields. Another was to identify farm fields that had not used water for a certain number of consecutive years, which would invalidate the claim - a radical and contentious application of technology which would have been dangerous to release without a fuller understanding of the politics and local contexts. Even with some of these databases built, we learned that water rights require handshakes and hat tips and we were trying to apply a scalable solution to a problem that wasn't ready to "scale." So we switched gears back to our compliance monitoring.

We met with The Nature Conservancy's {^Bird Returns} program via an introduction from our advisor Brian Richter, the then Director of International Water at The Nature Conservancy. The program applies management guidelines to rice fields in the Pacific Flyway, providing vital habitat for migratory birds in the Spring and Fall. After discussion with the team's leads, Katie Andrews and Ethan Inlander, we determined that our previous compliance monitoring software could be used to monitor projects in the BirdReturns portfolio. We launched this program, and over time it evolved into what is now Lens.

The team visited a project location in the Sacramento area The team visited a project location in the Sacramento area

In parallel, we met Gia Schneider from Natel Energy via a meandering series of introductions. Natel Energy developes a hydropower turbine that uniquely allows for 100% safe fish passage. After breakfast together in Boston, we realized we were working on the same underlying problems from different angles: Natel Energy via hardware and project development; Upstream Tech via software solutions - both with the aim at improving the health of watersheds and building climate resilience. After a few months of discussion and brainstorming, we decided to {join forces}. Learning about hydropower via Gia and the Natel team, we realized there was an opportunity to apply our machine learning and satellite technology to watershed management and planning, and more specifically starting with hydropower operations. This was the birth of our second product, HydroForecast.

Natel Natel's hanger in Alameda, CA

During this time period, we were fortunate to be joined by many friends, past colleagues, and people we met along the way. The team is now over {^20 people}.

⇣ (19)
  • I had seen this video years ago, but returned to it after talking to Tom today about my work at {Upstream Tech}. I feel like the clickbait title doesn't do it justice. There are some fantastic ideas in here that have guided how I approach learning.
  • 2021-06-01 
  • {Upstream Tech}
    • Meetings
      • 1-on-1s with Daniel, Kristi, Fiona
      • HF, Lens G&P and product syncs
      • Customer meeting
    • Intentions
      • Start HRRR series processor
      • Create SNODAS SWE visual example
    • HRRR
      • {^AWS Open Data Listing}
      • {^HRRR grib format bucket browser}
      • Zarr
        • {^HRRR zarr format bucket browser}
        • {^Zarr documentation}
        • "This archive contains sets for each model run of analysis and forecast files sectioned into 96 small chunks for every variable. Files within the AWS hrrrzarr S3 bucket are named to emulate a hierarchical data structure."
        • "Not all HRRR GRIB2 files are expected to be processed into Zarr format. Many users will find the GRIB2 format to be adequate for their needs. The types of use cases relevant for the Zarr archive require surface sensible weather parameters or meteorological parameters at "standard" levels in the vertical."
        • Variables at standard levels in the "sfc" bucket
        • "HRRR CONUS analysis (F00) files, whether for sfc or prs files, are stored in 96 'chunks' each containing 150x150 grid points. HRRR CONUS forecast (F01-FXX) files are stored in 96 3-D cubes (XX,150,150) where the forecast duration, FXX, depends on HRRR version and time of day. For example, V4 forecasts are available out to XX=48h at 00, 06, 12, 19 UTC and out to XX=18h at other hours of the day for variables produced in each forecast GRIB2 file."
        • Zarr files are typically available 3 hours after the initialization time, e.g., 00UTC analysis and forecast files are available by 03UTC
        • Available in Zarr:
          • V4 near real time
          • V4 2020-12-02 to current
          • V3 2018-07-12 to 2020-12-01
          • The dataset otherwise goes back to 2014-09-30, but would presumably need to be loaded via the grib datasets.
  • 2021-06-02 
  • {Upstream Tech} Intentions
    • Meetings
      • Prospective meeting on forecasting temp and DO
      • Amadou 1x1
    • Intentions
      • Advisory board planning
      • HRRR implementation
  • 2021-06-07 
  • {Upstream Tech}
    • TODO
      • HF sales outreach (all the pipedrive activities)
      • Wrap HRRR series processor
      • Map out Lens self-service flows
      • Email ketchup
    • Meetings
      • Product & sales kick-off syncs
      • Customer walkthrough call
  • 2021-06-08 
  • {Upstream Tech} Intentions
    • TODO
      • HF sales outreach (all the pipedrive activities)
      • HF sales contracting
      • HRRR test failures
      • Email ketchup
      • Proofread seasonal blog
      • Hiring cleanup and outreach
      • Review 15Fives
      • Check with Kathryn about end of internship timing
    • Meetings
      • Daniel, Mostafa 1x1
      • DOE conduit lunch and learn
  • 2021-06-09 
  • {Upstream Tech} Intentions
    • TODO
      • Email ketchup
      • Marketing site updates
      • Zapier video walkthrough and docs
      • Plan webinar
      • Review regional outlook sketch
    • Meetings
      • Kathryn 1-on-1
      • Call with full stack candidate
  • 2021-06-10 
  • {Upstream Tech} Intentions
    • TODO
      • Review Phil's HRRR changes
      • Start scoping veg clearing change detection
      • Plan webinar
    • Meetings
      • Abby, Lauren, Sam, Jeremiah 1x1s
    • Webinar outline
      1. welcome, provide outline
        • First I'll introduce myself and the team
        • I'll give a background of HydroForecast and the technology we use.
        • Then we are going to look at two of HydroForecast's live forecasts.
          • First the inflows at a dam in Idaho's North Fork Clearwater watershed during freshet.
          • Then we will look at the Trinity River basin in California, where low flows reflect an emerging drought.
      2. intro team and myself
      3. intro lens and hydroforecast
      4. Run through "What is HydroForecast?" slides
      5. Switch to basin deep-dives
  • 2021-06-11 
  • {Upstream Tech} Intentions (all meetings today!)
    • Laura, Camilla 1x1s
    • HF planning sync
    • Lens planning sync
    • HF webinar prep
    • Climate.AI tech meet (moved)
  • 2021-06-15 
  • {Upstream Tech} Intentions
    • TODO
      • Note count on property overview
    • Meetings
      • Hiring Natel GM coordination
      • Change detection brainstorm
      • HF customer demo
      • Kristi 1x1
      • Candidate interview
      • Conversation with Sweet Sensors
  • 2021-06-16 
  • {Upstream Tech} Intentions
    • TODO
      • Investigate imagery availability
    • Meetings
      • Hiring candidate conversation
      • Ag customer contracting
      • Webinar!
  • 2021-06-17 
  • {Upstream Tech} Intentions
    • TODO
      • Wrap up note count in property list
    • Meetings
      • Gia, Lauren, Carolyn, David 1x1s
      • Interview with Hydro GM candidate for Natel
      • Customer call
      • Hiring check-in
  • 2021-06-21 
  • Intentions ({Upstream Tech})
    • TODO
      • Jot down Natel candidate interview notes
      • Finish note count MR
    • Meetings
      • 1x1 Jeremiah, Dana
      • Team planning meetings
  • 2021-06-29 
  • {Upstream Tech} intentions
    • TODO
      • Candidate follow ups
      • Email ketch
      • Sketch HF outlook concept
      • Catch up on pipedrive activities
      • Maya comp!
    • Meetings
      • HF G&P sync
      • Daniel, Kristi 1x1s
  • 2021-06-30 
  • {Upstream Tech} intentions
    • TODO
      • Create animation for NF Shoshone basin
      • Look at potential improvements for extended archive ordering UX
      • HF outlook mock
      • Sales outreach and continued email ketch
  • 2021-07-19 
  • {Upstream Tech} intentions
    • Inbox 0 before vacation
    • Help HF broad outreach
    • 4 interviews
    • 1x1 with Maya
  • Now 
    Growing the {Upstream Tech} team this Spring and Summer

    I make {short songs}, am {learning to grow living things}, run a climate and conservation company {Upstream Tech}, and have a number of technical and non-technical {projects}, hobbies and interests. I used to be a strong rock climbing -- you could find me in the local rock gym on most days; these days you can find me in {the Middlesex Fells} running on the trails.

    5-year gestalt 

    I suppose 5 years ago was the start of {Upstream Tech}. If my next 5 years can be as productive and "successful" (whatever that vague measure means), I'll be glad. I've been fortunate that the past 5 years of work have brought a variety pack of different roles and responsibilities. That miscellany has been valuable in starting to understand the kinds of work, from minutiae to big picture stuff, is most rewarding, and therefore what my craft, in the lifelong sense, could be.

    An appeal for impact 

    If you want to be more direct, find an organization tackling the toughest climate problems. Look at {^the members of Greentown Labs}, {^the previous classes of Imagine H2O} and non-profits like {^The Nature Conservancy} and {^Trout Unlimited}— they need technology to push forward. I strongly recommend Bret Victor’s webpage {^What can a technologist do about climate change?} which in part inspired me to quit my job to {work on these problems}.