https://mixlabs.org/
Share Dialog
Share Dialog
https://mixlabs.org/
Subscribe to MixLabs
Subscribe to MixLabs
<100 subscribers
<100 subscribers
This experimental environment consists of two types of canisters: ISP and Buckets. There are three canisters in the experiment, including one ISP which is responsible for distributing blob files to subordinate Buckets and two Buckets which is responsible for receiving blob files sent by ISP and storing them in stable memory. According to the order of creation, these buckets are called Bucket#1 and Bucket#2.
Suppose a user continuously transmits files to ISP with a total size of about 12.5GB, each file does not exceed UPLOAD_SIZE_LIMITED (about 2MB). ISP distributes the Blob file to Bucket#1 in the beginning. When the stable memory usage of Bucket#1 exceeds RETIRING_THRESHOLD (about 5.5GB), ISP automatically distributes subsequent files to the newly created Bucket#2.
In the experiment, ISP will transmits 440 epochs of files, and sends 50 blob files in each epoch, so there are 22,000 blob files. Bucket#1 receives the first 234 epochs (11700 blob files), while Bucket#2 receives the next 206 epochs (10300 blob files).
This section shows the line graphs of the RTS parameters of ISP, Bucket#1 and Bucket#2 during the whole experiment. It is specially explained here that RTS Memory Size, RTS Heap Size and RTS Max Live Size use Kilobyte as the ordinate unit. RTS Total Allocation, RTS Reclaimed and Stable Memory Size use MByte as the ordinate unit.
Among the three line graphs of RTS Memory Size, Heap Size and Max Live Size, Figure ISP has the features:
RTS Memory Size will step after a period of several epochs, and the upper limit of RTS Heap Size will also follow the step. As for the reason for the step, we are not clear, and we need to further study the Wasm Memory to get a reasonable guess.
The lower limit of RTS Heap Size is approximately equal to RTS Max Live Size, which means that ISP hardly uses Heap Memory (about 2KB) after a GC.
Figures Bucket#1, Bucket#2 have the following differences from Figure ISP:
The lower limit of RTS Heap Size is not approximately equal to RTS Max Live Size, which means that Bucket always occupies more Heap Memory, and as the storage process continues to increase, it increases from about 1000KB at the beginning to 4000KB at the end
The timing and number of steps of RTS Memory Size of two Buckets do not seem to be correlated with ISP.
The meaning of each parameter in the line graphs below will be explained in the following sections.
RTS Memory Size is the current size of the wasm memory array (mostly the same as the canister memory size). This can only grow, never shrink in Wasm, even if most of it is unused.
In Overview section, we have mentioned some features of RTS Memory Size: RTS Memory Size will step after a period of several epochs, and the timing and number of steps of RTS Memory Size of two Buckets do not seem to be correlated with ISP.
In Figure 10, 12 and 14, there are the line graphs of absolute difference of RTS Memory Size. We can find the following features:
RTS Memory Size is a multiple of 64KB because WASM_PAGE_SIZE = 65536
RTS Memory Size only increases by one wasm page most of the time
RTS Heap Size contains the actual size of the current Motoko heap, i.e., the heap memory in use, and RTS Max Live Size, which is the largest heap size that has remained so far after a GC.
In Overview section, we have already mentioned some features of RTS Heap Size: when a step occurs in RTS Memory Size, the upper limit of RTS Heap Size will also step up. We have listed the difference between upper and lower limits of ISP and Buckets at each stage.
ISP：40KB、1800KB、3500KB、5200KB、7000KB
Bucket#1：0.75KB、1175KB、2350KB、3550KB、4700KB
Bucket#2：0.75KB、1175KB、2350KB、3550KB
RTS Max Live Size is the largest heap size that has remained so far after a GC.
The RTS Max Live Size in ISP is roughly around 40KB, while the RTS Max Live Size in Bucket steps with the step of RTS Memory Size, we list the approximate values of each stage of Buckets
Bucket#1：700Byte、1700Byte、2600Byte、3700Byte、4700Byte
Bucket#2：700Byte、1700Byte、2600Byte、3700Byte
It should be noted that, although the absolute difference of RTS Max Live Size has several steps, the line graph still looks like a straight line. After checking the log data, we find that RTS Max Live Size will increase by one more index interval after one more step. During the interval, RTS Max Live Size won’t increase.
RTS Total Allocation is the size of accumulative total bytes allocated.
The size of RTS Total Allocation of each epoch of ISP is roughly distributed around 240MB, while the size of RTS Total Allocation of each index of Bucket keeps increasing.
Similar to RTS Max Live Size, although the upper size of RTS Total Allocation of each index of Bucket keeps increasing, the line graph still looks like a straight line. We found the same reason as RTS Max Live Size, RTS Total Allocation will increase by one more index interval after one more step. RTS Total Allocation does not allocate any bytes in the interval, i.e., the size of allocated bytes is 0.
RTS Reclaimed is the size of accumulative total bytes collected.
The size of RTS Reclaimed of each epoch of ISP is roughly distributed around 80MB, while the size of RTS Reclaimed in Bucket steps with the step of RTS Memory Size,
Similar to RTS Max Live Size, although the upper size of RTS Reclaimed of each index of Bucket has several steps, the line graph still looks like a straight line. We found the same reason as RTS Max Live Size, RTS Reclaimed will increase by one more index interval after one more step. RTS Reclaimed does not collect any bytes in the interval, i.e., the size of collected bytes is 0.
Since Bucket is responsible for actually writing data, Stable Memory Size is only related to Bucket.
During the experiment, the stable memory writing speed of Bucket remained constant, each index was written to 597,324 Bytes.
Based on the analysis of the above line graphs, it can be concluded that the growth of Canister's RTS parameters are predictable, which has guiding significance for us to design a new generation of ISP architecture in the future.
In addition, it can be intuitively seen from the line graphs that the status of Bucket's continuous writing data to Stable Memory is robust and stable, which provides confidence for ISP to support larger-scale business services.
Motoko, Array, Memory - rossberg
As for the
rtsvalues:memory_sizeis the current size of the Wasm memory array (mostly the same as the canister memory size). This can only grow, never shrink in Wasm, even if most of it is unused.
The other values come from here 19. The two interesting ones are probably
heap_size, which contains the actual size of the current Motoko heap, i.e., the heap memory in use, andmax_live_size, which is the largest heap size that has remained so far after a GC. The other numbers are accumulative total bytes allocated and collected so far, which probably is less interesting.
motoko/rts/motoko-rts/src/memory/ic.rs
https://github.com/dfinity/motoko/blob/master/rts/motoko-rts/src/memory/ic.rs
/// Maximum live data retained in a GC. pub(crate) static mut MAX_LIVE: Bytes<u32> = Bytes(0);
/// Amount of garbage collected so far. pub(crate) static mut RECLAIMED: Bytes<u64> = Bytes(0);
/// Counter for total allocations pub(crate) static mut ALLOCATED: Bytes<u64> = Bytes(0);
Mixlabs is a leading-edge technology laboratory in the blockchain direction composed of top universities and community developers in Asia. It is mainly engaged in cutting-edge technology research, incubation and ecological support around Web3.
This experimental environment consists of two types of canisters: ISP and Buckets. There are three canisters in the experiment, including one ISP which is responsible for distributing blob files to subordinate Buckets and two Buckets which is responsible for receiving blob files sent by ISP and storing them in stable memory. According to the order of creation, these buckets are called Bucket#1 and Bucket#2.
Suppose a user continuously transmits files to ISP with a total size of about 12.5GB, each file does not exceed UPLOAD_SIZE_LIMITED (about 2MB). ISP distributes the Blob file to Bucket#1 in the beginning. When the stable memory usage of Bucket#1 exceeds RETIRING_THRESHOLD (about 5.5GB), ISP automatically distributes subsequent files to the newly created Bucket#2.
In the experiment, ISP will transmits 440 epochs of files, and sends 50 blob files in each epoch, so there are 22,000 blob files. Bucket#1 receives the first 234 epochs (11700 blob files), while Bucket#2 receives the next 206 epochs (10300 blob files).
This section shows the line graphs of the RTS parameters of ISP, Bucket#1 and Bucket#2 during the whole experiment. It is specially explained here that RTS Memory Size, RTS Heap Size and RTS Max Live Size use Kilobyte as the ordinate unit. RTS Total Allocation, RTS Reclaimed and Stable Memory Size use MByte as the ordinate unit.
Among the three line graphs of RTS Memory Size, Heap Size and Max Live Size, Figure ISP has the features:
RTS Memory Size will step after a period of several epochs, and the upper limit of RTS Heap Size will also follow the step. As for the reason for the step, we are not clear, and we need to further study the Wasm Memory to get a reasonable guess.
The lower limit of RTS Heap Size is approximately equal to RTS Max Live Size, which means that ISP hardly uses Heap Memory (about 2KB) after a GC.
Figures Bucket#1, Bucket#2 have the following differences from Figure ISP:
The lower limit of RTS Heap Size is not approximately equal to RTS Max Live Size, which means that Bucket always occupies more Heap Memory, and as the storage process continues to increase, it increases from about 1000KB at the beginning to 4000KB at the end
The timing and number of steps of RTS Memory Size of two Buckets do not seem to be correlated with ISP.
The meaning of each parameter in the line graphs below will be explained in the following sections.
RTS Memory Size is the current size of the wasm memory array (mostly the same as the canister memory size). This can only grow, never shrink in Wasm, even if most of it is unused.
In Overview section, we have mentioned some features of RTS Memory Size: RTS Memory Size will step after a period of several epochs, and the timing and number of steps of RTS Memory Size of two Buckets do not seem to be correlated with ISP.
In Figure 10, 12 and 14, there are the line graphs of absolute difference of RTS Memory Size. We can find the following features:
RTS Memory Size is a multiple of 64KB because WASM_PAGE_SIZE = 65536
RTS Memory Size only increases by one wasm page most of the time
RTS Heap Size contains the actual size of the current Motoko heap, i.e., the heap memory in use, and RTS Max Live Size, which is the largest heap size that has remained so far after a GC.
In Overview section, we have already mentioned some features of RTS Heap Size: when a step occurs in RTS Memory Size, the upper limit of RTS Heap Size will also step up. We have listed the difference between upper and lower limits of ISP and Buckets at each stage.
ISP：40KB、1800KB、3500KB、5200KB、7000KB
Bucket#1：0.75KB、1175KB、2350KB、3550KB、4700KB
Bucket#2：0.75KB、1175KB、2350KB、3550KB
RTS Max Live Size is the largest heap size that has remained so far after a GC.
The RTS Max Live Size in ISP is roughly around 40KB, while the RTS Max Live Size in Bucket steps with the step of RTS Memory Size, we list the approximate values of each stage of Buckets
Bucket#1：700Byte、1700Byte、2600Byte、3700Byte、4700Byte
Bucket#2：700Byte、1700Byte、2600Byte、3700Byte
It should be noted that, although the absolute difference of RTS Max Live Size has several steps, the line graph still looks like a straight line. After checking the log data, we find that RTS Max Live Size will increase by one more index interval after one more step. During the interval, RTS Max Live Size won’t increase.
RTS Total Allocation is the size of accumulative total bytes allocated.
The size of RTS Total Allocation of each epoch of ISP is roughly distributed around 240MB, while the size of RTS Total Allocation of each index of Bucket keeps increasing.
Similar to RTS Max Live Size, although the upper size of RTS Total Allocation of each index of Bucket keeps increasing, the line graph still looks like a straight line. We found the same reason as RTS Max Live Size, RTS Total Allocation will increase by one more index interval after one more step. RTS Total Allocation does not allocate any bytes in the interval, i.e., the size of allocated bytes is 0.
RTS Reclaimed is the size of accumulative total bytes collected.
The size of RTS Reclaimed of each epoch of ISP is roughly distributed around 80MB, while the size of RTS Reclaimed in Bucket steps with the step of RTS Memory Size,
Similar to RTS Max Live Size, although the upper size of RTS Reclaimed of each index of Bucket has several steps, the line graph still looks like a straight line. We found the same reason as RTS Max Live Size, RTS Reclaimed will increase by one more index interval after one more step. RTS Reclaimed does not collect any bytes in the interval, i.e., the size of collected bytes is 0.
Since Bucket is responsible for actually writing data, Stable Memory Size is only related to Bucket.
During the experiment, the stable memory writing speed of Bucket remained constant, each index was written to 597,324 Bytes.
Based on the analysis of the above line graphs, it can be concluded that the growth of Canister's RTS parameters are predictable, which has guiding significance for us to design a new generation of ISP architecture in the future.
In addition, it can be intuitively seen from the line graphs that the status of Bucket's continuous writing data to Stable Memory is robust and stable, which provides confidence for ISP to support larger-scale business services.
Motoko, Array, Memory - rossberg
As for the
rtsvalues:memory_sizeis the current size of the Wasm memory array (mostly the same as the canister memory size). This can only grow, never shrink in Wasm, even if most of it is unused.
The other values come from here 19. The two interesting ones are probably
heap_size, which contains the actual size of the current Motoko heap, i.e., the heap memory in use, andmax_live_size, which is the largest heap size that has remained so far after a GC. The other numbers are accumulative total bytes allocated and collected so far, which probably is less interesting.
motoko/rts/motoko-rts/src/memory/ic.rs
https://github.com/dfinity/motoko/blob/master/rts/motoko-rts/src/memory/ic.rs
/// Maximum live data retained in a GC. pub(crate) static mut MAX_LIVE: Bytes<u32> = Bytes(0);
/// Amount of garbage collected so far. pub(crate) static mut RECLAIMED: Bytes<u64> = Bytes(0);
/// Counter for total allocations pub(crate) static mut ALLOCATED: Bytes<u64> = Bytes(0);
Mixlabs is a leading-edge technology laboratory in the blockchain direction composed of top universities and community developers in Asia. It is mainly engaged in cutting-edge technology research, incubation and ecological support around Web3.
/// Heap pointer pub(crate) static mut HP: u32 = 0;
/// Heap pointer after last GC pub(crate) static mut LAST_HP: u32 = 0;
// Provided by generated code extern "C" { pub(crate) fn get_heap_base() -> u32; pub(crate) fn get_static_roots() -> Value; }
pub(crate) unsafe fn get_aligned_heap_base() -> u32 { // align to 32 bytes ((get_heap_base() + 31) / 32) * 32 }
#[no_mangle] unsafe extern "C" fn init(align: bool) { HP = if align { get_aligned_heap_base() } else { get_heap_base() }; LAST_HP = HP; }
#[no_mangle] unsafe extern "C" fn get_heap_size() -> Bytes<u32> { Bytes(HP - get_aligned_heap_base()) }
/// Provides a Memory implementation, to be used in functions compiled for IC or WASI. /// The Memory implementation allocates in Wasm heap with Wasm memory.grow instruction. pub struct IcMemory;
impl Memory for IcMemory { #[inline] unsafe fn alloc_words(&mut self, n: Words<u32>) -> Value { let bytes = n.to_bytes(); // Update ALLOCATED let delta = u64::from(bytes.as_u32()); ALLOCATED += Bytes(delta);
// Update heap pointer
let old_hp = u64::from(HP);
let new_hp = old_hp + delta;
// Grow memory if needed
grow_memory(new_hp);
debug_assert!(new_hp <= u64::from(core::u32::MAX));
HP = new_hp as u32;
Value::from_ptr(old_hp as usize)
}
}
/// Page allocation. Ensures that the memory up to, but excluding, the given pointer is allocated. #[inline(never)] unsafe fn grow_memory(ptr: u64) { debug_assert!(ptr <= 2 * u64::from(core::u32::MAX)); let page_size = u64::from(WASM_PAGE_SIZE.as_u32()); let total_pages_needed = ((ptr + page_size - 1) / page_size) as usize; let current_pages = wasm32::memory_size(0); if total_pages_needed > current_pages { if wasm32::memory_grow(0, total_pages_needed - current_pages) == core::usize::MAX { rts_trap_with("Cannot grow memory"); } } }
/// Heap pointer pub(crate) static mut HP: u32 = 0;
/// Heap pointer after last GC pub(crate) static mut LAST_HP: u32 = 0;
// Provided by generated code extern "C" { pub(crate) fn get_heap_base() -> u32; pub(crate) fn get_static_roots() -> Value; }
pub(crate) unsafe fn get_aligned_heap_base() -> u32 { // align to 32 bytes ((get_heap_base() + 31) / 32) * 32 }
#[no_mangle] unsafe extern "C" fn init(align: bool) { HP = if align { get_aligned_heap_base() } else { get_heap_base() }; LAST_HP = HP; }
#[no_mangle] unsafe extern "C" fn get_heap_size() -> Bytes<u32> { Bytes(HP - get_aligned_heap_base()) }
/// Provides a Memory implementation, to be used in functions compiled for IC or WASI. /// The Memory implementation allocates in Wasm heap with Wasm memory.grow instruction. pub struct IcMemory;
impl Memory for IcMemory { #[inline] unsafe fn alloc_words(&mut self, n: Words<u32>) -> Value { let bytes = n.to_bytes(); // Update ALLOCATED let delta = u64::from(bytes.as_u32()); ALLOCATED += Bytes(delta);
// Update heap pointer
let old_hp = u64::from(HP);
let new_hp = old_hp + delta;
// Grow memory if needed
grow_memory(new_hp);
debug_assert!(new_hp <= u64::from(core::u32::MAX));
HP = new_hp as u32;
Value::from_ptr(old_hp as usize)
}
}
/// Page allocation. Ensures that the memory up to, but excluding, the given pointer is allocated. #[inline(never)] unsafe fn grow_memory(ptr: u64) { debug_assert!(ptr <= 2 * u64::from(core::u32::MAX)); let page_size = u64::from(WASM_PAGE_SIZE.as_u32()); let total_pages_needed = ((ptr + page_size - 1) / page_size) as usize; let current_pages = wasm32::memory_size(0); if total_pages_needed > current_pages { if wasm32::memory_grow(0, total_pages_needed - current_pages) == core::usize::MAX { rts_trap_with("Cannot grow memory"); } } }
No activity yet