jonnifyelixir
F2 / F2.08 / pipeline

F2.08 · part 3 of 3

Building pipelines

A pipeline strings the higher-order functions together: a value enters, each stage transforms it, and the last stage hands back the result. Read top to bottom, the whole transformation is one legible sequence — map to reshape, filter to select, reduce to combine.

This is how most real Elixir reads. The data starts as a collection; each stage is a small, independent step; nothing is mutated, so every intermediate is a value you could inspect. Building a pipeline is choosing the stages and their order — the same composition as before, now over collections.

Stage by stage A string pipeline Worked examples

Stage by stage

Step through a numeric pipeline one stage at a time: start with a list, triple every element, keep those past nine, then sum what remains. Each stage shows the value it produces.

Advance the pipeline stage by stage.

pipeline
a sequence of stages a value flows through.
stage
one step — a map, a filter, a reduce.
intermediate
the value between two stages; with immutability, always inspectable.
end to end
from the input collection to the final result.

map → filter → sum · one stage at a time

0 of 3
[1, 2, 3, 4, 5, 6] |> map(&1 * 3) |> filter(> 9) |> sum() 


      
nothing applied yet · the input is [1, 2, 3, 4, 5, 6]

    

    

      

        
Idea

        
Each stage is independent; the value between any two is a concrete list.

      

      
      

        
F2.05 · folds

        
map, filter, and sum are the folds this pipeline is assembled from.

      

    

    
A pipeline transforms a value one stage at a time. Every intermediate is itself an inspectable
      value.

  


  
  

    
A string pipeline

    

      
Pipelines are not only for numbers. Split a sentence into words, keep the longer ones, upcase them, and
        join them back — four stages over text. Set the minimum word length and watch the stages adjust.

      
Set the minimum length to keep.

    


    

      
split → filter → upcase → join

      

        

          
          
          4
        

      


      
        |> String.split()
        ["the", "quick", "brown", "fox"]
        |> filter(len ≥ k)
        ["quick", "brown"]
        |> map(upcase)
        ["QUICK", "BROWN"]
        |> join(" ")
        
        "QUICK BROWN"
      

      


      
keep words ≥ 4 letters · 2 of 4 words kept · "QUICK BROWN"

    

    

      

        
Idea

        
The same stages compose over any collection — words as readily as numbers.

      

      
      

        
F2.09 · the lab

        
The lab runs a longer pipeline over a real dataset, stage by stage.

      

    

    
Pipelines compose over any data. Four small stages turn a sentence into a filtered, upcased
      line.

  


  
  

  
  

    
Where this goes

    

      
That completes F2.08. Composition feeds one function’s output into the next; the pipe operator writes
        that chain in reading order; and a pipeline is composition over a collection, each stage a small step the
        value flows through. With map, filter, and reduce as the stages, a transformation reads as a top-to-bottom
        sequence. F2.09, the data-pipeline lab, brings all of F2 together over a real dataset — watching the value
        change at every stage.

    

    
Next: F2.09 — The data-pipeline lab — compose
      map, filter, and reduce over a dataset and watch each stage. Back to the
      F2.08 hub or the F2 overview.