Machine learning (ML) has gained attention from the network research community because it can help solve difficult problems and potentially lead to groundbreaking achievements. In the Wi-Fi domain, ML is applied to solve challenges such as efficient channel access and fair coexistence with other technologies in unlicensed bands. In this paper, we address the performance of uplink orthogonal frequency division multiple random access (UORA) in IEEE 802.11ax networks. Optimization of UORA is a good case for applying ML because of its inherent complexity and dependence on situation and time-dependent parameters. In particular, we use deep reinforcement learning to tune UORA parameters. Our simulation results show that even though the ML-based solution leads to close to optimal results, its operation is comparable to a much simpler, non-ML heuristic. Therefore, we conclude that ML-based solutions to improve IEEE 802.11 performance need not exceed well-designed heuristics.