How to convert an iterator of [u8; 4] into an iterator of u8?

Question

I have u8 data that I need to expand into more bytes, [u8; 4] [u8; 4] via a function and pass as an iterator in its entirety into a second function that consumes it.

The following compiles but doesn't work because it calls the consuming function multiple times:

fn expand(b: u8) -> [u8; 4] {
    const T: [u8; 4] = [0x12, 0x34, 0x56, 0x78];
    [
        T[((b >> 6) & 0b11) as usize],
        T[((b >> 4) & 0b11) as usize],
        T[((b >> 2) & 0b11) as usize],
        T[((b >> 0) & 0b11) as usize],
    ]
}

fn process2(data: impl Iterator<Item = [u8; 4]>) {
    for x in data {
        process(x.iter().cloned());
    }
}

fn process(data: impl Iterator<Item = u8>) { 
    for x in data {
        println!("{:02x}", x);
    }
}

fn main() {
    let xs = [1, 2, 3, 4];
    process2(xs.iter().map(|x| expand(*x)));
}

Playground

flat_map seems to be the answer but I run into lifetime problems:

    process(xs.iter().map(|x| expand(*x)).flat_map(|x| x.iter().cloned()));

gives me:

error[E0515]: cannot return value referencing function parameter `x`
  --> src/main.rs:27:56
   |
27 |     process(xs.iter().map(|x| expand(*x)).flat_map(|x| x.iter().cloned()));
   |                                                        -^^^^^^^^^^^^^^^^
   |                                                        |
   |                                                        returns a value referencing data owned by the current function
   |                                                        `x` is borrowed here

How do I convert Iterator<Item=[u8; 4]> Iterator<Item=[u8; 4]> into Iterator<Item=u8> ?

Answer 1

You can't using only the Rust 1.49 stable standard library.

Nightly

Use array::IntoIter :

// 1.52.0-nightly (2021-02-09 097bc6a84f2280a889b9)
use std::array;

fn example() -> impl Iterator<Item = u8> {
    let data: [u8; 4] = [1, 2, 3, 4];
    array::IntoIter::new(data)
}

Stable Rust

Use arrayvec :

use arrayvec::ArrayVec; // 0.5.2

fn example() -> impl Iterator<Item = u8> {
    let data: [u8; 4] = [1, 2, 3, 4];
    ArrayVec::from(data).into_iter()
}

Applying it

You can then use Iterator::flat_map with either solution:

use arrayvec::ArrayVec; // 0.5.2

fn example(data: [u8; 4]) -> impl Iterator<Item = u8> {
    ArrayVec::from(data).into_iter()
}

fn drive_it(i: impl Iterator<Item = [u8; 4]>) -> impl Iterator<Item = u8> {
    i.flat_map(example)
}

See also:

• How do I avoid allocations in Iterator::flat_map?

Answer 2

I would advise creating a custom iterator:

struct NameMe {
    i: u8,
    v: u8,
}

impl NameMe {
    fn new(v: u8) -> Self {
        Self { i: 8, v }
    }
}

impl Iterator for NameMe {
    type Item = u8;

    fn next(&mut self) -> Option<Self::Item> {
        const T: [u8; 4] = [0x12, 0x34, 0x56, 0x78];

        if self.i != 0 {
            self.i -= 2;

            Some(T[((self.v >> self.i) & 0b11) as usize])
        } else {
            None
        }
    }
}

fn main() {
    let xs = [1, 2, 3, 4];

    let result: Vec<_> = xs.iter().copied().flat_map(NameMe::new).collect();

    let expect = [
        0x12, 0x12, 0x12, 0x34, 0x12, 0x12, 0x12, 0x56, 0x12, 0x12, 0x12, 0x78, 0x12, 0x12, 0x34,
        0x12,
    ];

    assert_eq!(result, expect);
}