packets sometimes sent even if port not forwarded

When behind a router, we need to forward port, on the router, for the specific host listening on that port.

If we don't forward the port, it will not always work on the host.

But why not always ? Why can I still, sometimes, listen to that port on the host, even if the port is not forwarded on the router ?

Does the router guessing which host wants the packets ?