Rxjs add/remove timeout operator dynamically

hi, in my app, iam working with on a service, which i use to communicate over udp. Responses from udp communication are sent through subject which i created. Here is the method, for getting observable:

return this.udpSubject.asObservable().pipe(
      switchMap((response: any) => {
        return response.timeoutNeeded
          ? of(response).pipe(timeout(5000))
          : of(response);
      map((response: any) => {
        return {
          result: response.result,
      catchError(err => {
        this.closeUdpConnection(socketId, udpResponseHandler);
        return of({ result: "0", text: "Communication error" });

The problem is, that i want to apply timeout operator only if there is timeoutNeeded property on response object true. I used switchmap for that and it works great, but only for cases, when some response is emitted. But when there is no emit, the switchmap operator doesnt run, so it waits forever. For these cases i want to apply 5s timeout. How can i do this? Is it possible to reset timeout operator or remove it dynamically?

First thing I would try is cobbling something together with the timeoutWith operator.