Before the data
A dial-up modem had to establish a working connection over a telephone circuit before it could carry your internet traffic. The whistles and bursts were signals exchanged with the equipment at the other end. They were part of setting up that connection, rather than a sound effect added by the internet provider.
The familiar recording also contains sounds that precede the modem conversation. There is a dial tone, followed by the tones used to dial the telephone number. Only after the remote equipment answers can the two ends begin arranging how to communicate. A busy line could stop the process before that arrangement began.
Several jobs in one recording
Oona Räisänen analysed a recording made by William Termini and displayed its changing frequencies as a spectrogram. Her account separates the initial identification exchange from later testing of the circuit and adjustment of the receiving equipment. The modems establish a compatible transmission mode and work out what the line can support.
Some tones also deal with equipment elsewhere in the telephone network. Echo-control systems help ordinary speech, but can interfere with modem communication. RFC 4734 describes answer signals used to disable echo suppressors and cancellers. Changes within a sustained tone can therefore have a specific job even when they sound much like the rest of the noise.
That analysis describes a particular connection. Different modem standards have different startup procedures, and the exact sequence in one recording is not a universal script. A recognisable handshake does not mean every modem was using the same protocol or would settle on the same speed.
The speaker could stop listening for you
Räisänen also notes that the command ATM0 could switch off the modem’s speaker. This changed what the person at the desk heard. It did not remove the electrical signals that the modems needed to exchange over the telephone circuit.
The end of the audible sequence likewise did not mean that signalling had ended. Once startup was complete, the connection could carry data. The speaker and the communications link were separate parts of the experience: one made progress audible, while the other did the work.