How do we select the value for with_iops_counter for Throttle?

[debajyoti-truefoundry]

In this example, we see:

Throttle::new()
    .with_read_iops(4000)
    .with_write_iops(2000)
    .with_write_throughput(100 * 1024 * 1024)
    .with_read_throughput(800 * 1024 * 1024)
    .with_iops_counter(IopsCounter::PerIoSize(NonZeroUsize::new(128 * 1024).unwrap()))

I would like to understand how we select the value for PerIoSize in this example.
I am using 0.17.3.

[MrCroxx]

Hi.

The value is based on the IOPS counting model of your storage vendor.

e.g.

• If you are building the hybrid cache over your own disk, the PerIo counter is all you need.
• If you are building the byhrid cache over AWS EBS gp3 (or other EBS instances), AWS counts the IOPS per 128KiB (IIRC), use PerIoSize(128 * 1024) as the counter will give a more accurate throttle result.

[debajyoti-truefoundry]

In this case, obtain will return a miss before requesting any disk IO, the only overhead comes from the memory cache lookup miss.

Is it possible to know that the throttling was the reason behind the miss? In this case, I may not want to put the data back in the cache, as reading throttling will also result in write throttling.

Also, what happens if we hit the write throttle in insert API?

[MrCroxx]

as reading throttling will also result in write throttling

That won't be the case. foyer count read/write throttle separately.

Also, what happens if we hit the write throttle in insert API?

The entry to insert will be ignored.

[debajyoti-truefoundry]

as reading throttling will also result in write throttling
That won't be the case. foyer count read/write throttle separately.

I understand, but in my system, if obtain does not return anything, I consider that a cache miss, retrieve the data from my source, and then put it back in the cache. Now, if obtain does not return anything due to read throttling, I will attempt to reinsert the data into the cache, which can also throttle, as the IOPS and throughput limits are common across input and output operations for GP3.

[MrCroxx]

Emmm. Why not consider the fetch() API in this case?

I think the behaviour of fetch() API fits your needs:

• If the hybrid cache returns a miss not because of throttle, fetch from remote storage and fill the cache.
• If the hybrid cache returns a miss because of throttle, fetch from remote storage but don't fill the cache.

[debajyoti-truefoundry]

Oh nice! The fetch API already implement this behaviour?

I will check it out.