Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I think you're answering to the wrong comment. I'm saying calloc doesn't actually take milliseconds (which would have been absurd), so feel free to use it always.

Calling calloc/free on a 1 MB buffer seems to take about 50 microseconds on my computer (with no optimizations), and I consider that an unusually large allocation.



For an allocation of that size, it's likely that your system's malloc is requesting the allocation directly from the OS's virtual memory system with mmap(2), and freeing it with munmap(2), without ever actually writing a megabyte of zeros. Smaller allocations, however, are likely to require explicit clearing since they will hardly ever require a system call. (It depends on the details of your OS and malloc, of course.)


This is on Linux with glibc, and according to strace, mmap was called exactly once at startup for a ~1 MB buffer, then never again.


> Calling calloc/free on a 1 MB buffer seems to take about 50 microseconds on my computer (with no optimizations), and I consider that an unusually large allocation.

Do you actually touch it and write to it? If you don't the allocation may be provisional and not actually happen until you try to write memory, in which case depending on the system you'll either get a COW fault (IIRC linux points the allocation to a zeroed COW page, first time you need to write it'll copy the page and let you write in) or a zero-fill fault (e.g. freebsd doesn't allocate the BSS by default, write access will cause a zero-fill fault and in response the VM will actually allocate the page — which it can do from a queue of pre-zeroed pages)


> Do you actually touch it and write to it?

calloc itself calls memset in user space. That's what separates it from malloc. Is it so hard to believe a modern computer can memset 20 GB/s?

If I run "dd if=/dev/zero of=/dev/null bs=1M count=10000", I get 21.0 GB/s. Reading from /dev/zero is basically performing a memset to dd's buffer in kernel space. Writing to /dev/null is a no-op, so it's a very similar test.


> calloc itself calls memset in user space.

Have you considered reading my comment past the first phrase? If you don't actually touch the page, the system doesn't need to allocate it, let alone zero it. If you don't touch the page, assuming you're benching on linux you're measuring the cost of mapping a preexisting zeroed COW page, not the cost of actually allocating that memory.


Calling memset in userspace qualifies as touching the page, no?


calloc is a library function, as opposed to a system call, so it calling memset is no different from the caller doing so. calloc will do the write in "copy on write". Calling memset on every byte in a buffer should qualify as "touching it".

This is why I said "in user space".




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: