The aim of this paper is to consider the problem of unreliable statistical inference caused by the presence of statistical misspecification, and discuss the merits of alternative ways to address the problem like invoking generic robustness results or/and using nonparametric inference. For simplicity the discussion focuses on the t-test for hypotheses concerning the mean in the context of the simple Normal model, with the misspecification coming in the form of Markov Dependence (MD). By deriving explicitly the nominal and actual error probabilities, it is shown that the presence of MD turns the t-testinto an unreliable procedure. It is argued that invoking traditional robustness arguments can often be very misleading and, in general, this strategy does not address the unreliability of inference problem, even if one were to use the actual error probabilities. A more appropriate strategy is to respecify the original statistical model to account for the misspecification, and test the hypotheses of interest using an inference procedure that is optimal in the context of the respecified model. It is shown that the presence of MD gives rise to the Autoregression (AR(1)) as the respecified model, and one can test the original hypotheses concerning the mean. The optimal t-test in the context of the AR(1) is shown to be related but different from the original and the modified t-tests.