The establishment of a reference population for genomic selection in Korean beef cattle is an ongoing process. There is a high likelihood of sex-specific differences in the composition of the reference and test populations. This study evaluates the accuracy of Genomic Estimated Breeding Values (GEBVs) for carcass traits in Hanwoo cattle, specifically investigating the efficacy of cow-based reference populations. The effectiveness of genomic selection (GS) is heavily dependent on the composition and size of the reference population. Utilizing genotype data from a Hanwoo 50k SNP chip and phenotypic data from 19,168 steers and 6,233 cows, the study estimated GEBV accuracies for carcass weight (CWT), eye muscle area (EMA), backfat thickness (BF), and marbling score (MS) using the GBLUP method. Results demonstrate that steer-based reference populations achieved the highest accuracy (0.64–0.88), averaging 0.78, likely due to standardized management and higher trait heritability (0.39–0.51) compared to cows. In contrast, cow-based reference populations exhibited prediction accuracies (0.55–0.75) in four traits using adjusted residual phenotype, averaging 0.64, but remained highly practical alternatives. While growth traits (CWT and EMA) showed significant bias in cross-sex predictions, fat deposition traits (BF and MS) remained stable across sexes. The study concludes that although steer-based populations provide optimal accuracy, incorporating cows into the reference population is strategically vital in Hanwoo.