Structuring a Statistical Analysis with R Using Emacs/ESS

Question

I am looking for a way to structure my statistical analysis. I currently use Emacs/ESS and the analysis file turned out quite long. I have started to put parts of the code into sourceable functions in a separate file, but still...

I would like to introduce some kind of subheadings/ section titles in the file (Aggregation, Cluster-analysis, Simulations, ...) and write the code beneath it, so I could quickly jump to the sections that I want to work on.

I guess I could just use comments and search for them, but I couldn't get an overview or an index that way. I also thought about using Org-Mode for the Headlines, but that wouldn't be very convenient for collaborators, who work with another Editor.

I know R-Studio implements this with sections, so there will be a solution for emacs, right?

Thank you very much!

PS: something like imenu would work, but that's just for functions, not for sections

Answer 1

In general I use org-mode and org-babel, but when I've got to share scripts with others, I've got the following in my .emacs :

(defgroup ess-jb-faces nil
  "Faces used by cutomized ess-mode"
  :group 'faces)

(defface ess-jb-comment-face
  '((t (:background "cornsilk" 
    :foreground "DimGrey"
    :inherit font-lock-comment-face)))
  "Face used to highlight comments."
  :group 'ess-jb-faces)

(defface ess-jb-comment-bold-face
  '((t (:weight bold
    :inherit ess-jb-comment-face)))
  "Face used to highlight bold in comments."
  :group 'ess-jb-faces)

(defface ess-jb-h1-face
  '((t (:height 1.6 
    :weight bold 
    :foreground "MediumBlue"
    :inherit ess-jb-comment-face)))
  "Face used to highlight h1 headers."
  :group 'ess-jb-faces)

(defface ess-jb-h2-face
  '((t (:height 1.2 
    :weight bold 
    :foreground "DarkViolet"
    :inherit ess-jb-comment-face)))
  "Face used to highlight h2 headers."
  :group 'ess-jb-faces)

(defface ess-jb-h3-face
  '((t (:height 1.0 
    :weight bold 
    :foreground "DarkViolet"
    :inherit ess-jb-comment-face)))
  "Face used to highlight h3 headers."
  :group 'ess-jb-faces)

(defface ess-jb-hide-face
  '((t (:foreground "white"
    :background "white"
    :inherit ess-jb-comment-face)))
  "Face used to hide characters."
  :group 'ess-jb-faces)

(font-lock-add-keywords 'ess-mode
   '(("^###\\(#\\)\\([^#].*\\)$" (1 'ess-jb-hide-face t)(2 'ess-jb-h1-face t))
     ("^###\\(##\\)\\([^#].*\\)$" (1 'ess-jb-hide-face t)(2 'ess-jb-h2-face t))
     ("^###\\(###\\)\\([^#].*\\)$" (1 'ess-jb-hide-face t)(2 'ess-jb-h3-face t))
     ("^###\\( .*\\|$\\)" 1 'ess-jb-comment-face t)
     ("^###" "\\*.*?\\*" nil nil (0 'ess-jb-comment-bold-face append))
     ))

With this, any comment with #### at the beginning of the line is formatted as an "header 1". Any comment with ##### is formatted as an "header 2", etc. And lines which begin with ### is seen as a comment with a special font-lock (used for long comments).

This can give something like this :

This is quite hacky, but the advantage is that it only uses standard R comments and as such can be shared without problem with others. In general I use the following for "header 1" : others see it as below, while I enjoy my defined font-lock :

############################################
#### HEADER 1
############################################

With this syntax, you can also use the following to activate outline-minor-mode on the sections defined previously and be able to fold/unfold them :

(add-hook 'ess-mode-hook
      '(lambda ()
         (auto-complete-mode nil)
         (outline-minor-mode 1)
         (setq outline-regexp "\\(^#\\{4,6\\}\\)\\|\\(^[a-zA-Z0-9_\.]+ ?<- ?function(.*{\\)")
         (setq outline-heading-alist
           '(("####" . 1) ("#####" . 2) ("######" . 3)
             ("^[a-zA-Z0-9_\.]+ ?<- ?function(.*{" .4)))))

All this code has not been very well tested, and I'm far from an expert in emacs lisp, so there should be better ways to do it, and don't be surprised in case of bugs !

Answer 2

Recent (Feb 2013) additions to Orgmode mean that you should now be able to embed org headings in your source code, and then navigate through them using orgstruct-mode. So, upgrade your org mode via git, and then try opening the following example R file. When you are on a comment line that has an org heading embedded, just hit TAB, or shift-TAB, and you should get org-mode headings.

### * Create data
data = list( s1=list(x=1:3, y=3:1), 
     s2=list(x=1:5, y=1:5), s3=list(x=1:4, y=rep(3,4)))

### * Base graphics version


par(mfrow=c(2,2))
lapply(data, plot)

### * Lattice version

nplots <- length(data)
pts.per.plot <- sapply(data, function(l) length(l$x))
df <- data.frame(which=rep(1:nplots, times=pts.per.plot),
                 x=unlist(sapply(data, function(l) l$x)),
                 y=unlist(sapply(data, function(l) l$y)))

xyplot(y~x|which, data=df, layout=c(2,2))

### ** Make the pdf
pdf(file='o.pdf')
xyplot(y~x|which, data=df, layout=c(2,1))                 
dev.off()

### * End of file

### Local Variables:
### eval: (orgstruct-mode 1)
### orgstruct-heading-prefix-regexp: "### "
### End:

Answer 3

Sounds like you have one big analysis script. That's not very optimal...

Think about redoing it with RMarkdown or Sweave, and using knitr to run it. Then you can use emacs' tools for navigating markdown or TeX and you can use the caching system in knitr to save having to do analyses right from the start when you've changed something near the end.

You also get a formatted report out of the analysis, if you want.

Also, breaking stuff out into separate files for sourcing is a bit sub-optimal - far far better to write functions and put them in a package - and to use the devtools package to make working with it really simple. Just edit your .R files and load_all updates it (none of the complex package build stuff of times past).

Big winnage.

Answer 4

polymode provides R+markdown, R+brew, R+cpp and other stuff. It is still in early stage of development and seems to work reliably only on very recent emacs (24.3).

Answer 5

Similarly to the orgstruct-mode mentioned above you can also use the new outshine package which works together with outline-minor-mode to interpret comments with asterisks as org-mode-like headings. They can be folded/expanded/added/etc using simple shortcuts like TAB, etc.

Answer 6

Another option, which I've been happy with over the years is allout-mode. Using per-file local variables:

## Local variables:
## allout-layout: (-1 : 0)
## End: