There's a delay between the moment I play a note on the keyboard, and the moment I heard this note.

This delay is due to the latency of your soundcard.

To get a better latency you must use an ASIO compatible soundcard and its ASIO driver.

==> http://www.asio4all.com/