You want to display lines along the x- and y-axes, but not on the other sides of the graph.

Using themes, use axis.line (Figure 8-24):

library(gcookbook) # For the data set

p <- ggplot(heightweight, aes(x=ageYear, y=heightIn)) + geom_point()

p + theme(axis.line = element_line(colour="black"))

Figure 8-24. Left: scatter plot with axis lines; right: with theme_bw(), panel.border must also be made blank

If you are starting with a theme that has a border around the plotting area, like theme_bw(), you will also need to unset panel.border (Figure 8-24, right):

p + theme_bw() +
    theme(panel.border = element_blank(),
          axis.line = element_line(colour="black"))