Nin Labs

Continuing my series of posts on cognitive neuroscience of memory. In this post, I first give an overview of attentive memory, some examples of attentive memory failures, and then end with some thoughts on how we might want to support attentive memory in our design of interfaces and tools.

The frontal lobes is the most modern addition to the human brain, providing facilities for planning and reasoning about everyday tasks and social situations.  Within the prefrontal cortex (PFC), lies amazing circuity for maintaining attention on important items during a task.

Attentive Memory

Attentive memory holds conscious memories that can be freely attended to.
Within it, goals, plans, and task-relevant items can be sustained for substantial periods of time. Attentive memory is found in the ventrolateral and dorsolateral prefrontal cortex, a region situated in the anterior portion of the brain’s frontal lobe. One theory is that these regions provide the ability to maintain attention on modality-specific specific information such as visual targets in spatial locations or verbal information.

Attentive memory has two complementary operations with corresponding neural mechanisms: focusing and filtering.

Focusing

The ability to maintain focus on items has been well studied in the PFC of primates and humans. In early studies of monkey brains, when a food reward was shown to a monkey and then subsequently hidden for a delay period, persistent firing of neurons in the PFC was sustained during the delay period. Despite distracting stimuli, the monkey could recall the location of the food reward. However, monkeys with damage to the PFC could not maintain attention and performed poorly at recalling the location of the food [Fuster 71].

A human equivalent of these monkeys has been found in one patient, Clive Wearing. After a viral infection that destroyed much of his PFC and hippocampus, the patient could no longer attend to any memory longer than 30 seconds [Wearing 2006]. Definitely worth watching the documentaries: part 2a, part 2b,part 2c, part 2d; and more recently: “Man with the 30 second memory”.

Clive’s Diary:

Filtering

Humans can fluidly filter and switch among items in attentive memory using selective attention. Selective attention is the ability to conditionally attend to an item based on its attributes. For example, in a crowded room, attention can be switched from people with red shirts to people with hats. This ability derives from highly plastic neurons that can become tuned sensitize to certain attributes, such as color [Fuster 82]. Selective attention complements our ability to focus on multiple items by allowing distinct groups of items to be attended to. Thus, via selection attention, more items can be concurrently attended given that the items have distinct attributes [Park 2007].
Finally, dynamic filtering is a process that integrates information retrieved from auto-associative recall and filters out information not related to the active goal.

Neural Mechanisms

More recent work has uncovered the underlying mechanisms of attentive memory.
When examining the firing patterns of ensembles of neurons, rhythmic oscillations can be observed. These oscillations encode the attributes of an attended item. Siegel and colleagues [Siegel 2009] observed that when multiple items need to be attended to, distinct items were maintained in distinct phase orientations of the oscillating signal. Our limited ability to attend to multiple items are constrained by the speed and space it takes to encode waves within a limited frequency spectrum. An interesting benefit emerging from phase coding of items is the “free” temporal order of those items. In the same experiment, when the order of items were misremembered, there was a correlation with inadequate phase separation of the encoded items: The signal still preserved enough information to represent the items, but not enough information was available to determine their order.

Why We Should Stop Using the Term “Working Memory”

Working memory is a conflated term based on outdated science.

The term working memory was first introduced in 1960 in Miller and colleagues’ influential book [Miller 60] on plans and reasoning as a system for temporary retention of plans and goals. The context of usage was for reasoning processes in cognitive models without strong considerations with concerns such as memory duration or task switching.

When Baddeley and Hitch first introduced their working memory model, they were concerned with the unitary view of short-term memory [Baddeley 2003]. What they wanted to emphasize was that short-term memory was not a passive store, but was interleaved with distinct modality-based processing of sensory input (separate verbal and visual processing). Further, they wanted to consider the low-level attentional processes needed to maintain those memories (over the time scale of seconds). What Baddeley describes can best be attributed to perceptual systems located in posterior regions of the brain. For example, patients with lesions to the left supermarginal gyrus (phonological loop) will have difficulty holding words in memory [Vallar 84]; whereas patients with a lesions in the right parieto-occiptial region have difficulty with spatial locations (for example, remembering a series of spots pointed to by another person).

Unfortunately, researchers across multiple disciplines have conflated the different usages of these terms to mean the same thing — that a perceptual store is the same as an “temporarily indefinite” pool of memory. That is, when referring to something such as the working memory of a programmer, a researcher is most likely meaning it in the sense of Miller and not Baddeley. Baddeley’s working memory theory cannot account for how people perform complex tasks or recover from interruptions. Theories such as long-term working memory attempt to reconcile this difference, but offer no neuroscience basis for the theory [Ericsson 95]; leaving some researchers to suggest the standard model of working memory has outlived its usefulness [Postle 2006].

Finally, recent direct evidence in the brain suggest that activity in neurons in the PFC are related to attended items and not remembered ones [Lebedev 2004].

Programmers and Attentive Memory

What are attentive memory failures that workers like programmers face?

Some programming tasks require developers to make many changes across a codebase. For example, if a developer needs to refactor code in order to move a component from one location to another or to update the code to use a new version of an API, then that developer needs to systematically and carefully edit all those locations affected by the desired change. Unfortunately, even a simple change can lead to many complications, requiring the developer to track the status of many locations in the code. Even worse, after an interruption to such as task, the tracked statuses in attentive memory quickly evaporate and the numerous visited and edited locations confound retrieval.

Studies examining refactoring practices of programmers have found several deficiencies in tool support [Murphy-Hill 11],[Mohsen 12]. One essential deficiency is the lack of ability to track the statuses of many locations in code. As a work-around, developers abandon refactoring tools and instead rely on compile errors that were introduced when refactoring. Interactive compile errors (which appear or disappear automatically as a programmer makes changes) can represent the task well in an automated fashion: A correct change removes the compile error, whereas an incorrect change arising from a complication adds more compile errors. A programmer can be interrupted in this state and still have a means to continue the task. Unfortunately, using compile errors to track changes is not a general solution and can still lead to incorrect refactorings [Xi 12].

Supporting Attentive Memory

Given the volatile and limited nature of attentive memory, how can we better support it in our tools and interfaces? In general, a tool should provide a mechanism for a persisted and stateful focus, on a numerous items, with various modalities.

In programming environments, code tabs are a central but limited interface element for supporting attentive memory [Ko 2006]. Bookmarks are another supportive but rarely used device. Arguably, Mylyn offers a good start. It is a programming tool that offers attentive memory support, by providing an ability to focus and filter programming files based on those that were visited or edited during a task. However, there are still several short-comings. For example, the focus mechanism is binary (degree-of-interest aside), and does not include stateful properties such as status of an unresolved issue or whether an item is edited or visited (which I might want to know if I’m in the middle of making a change to hundreds of locations).

In desktop environments, the Task Bar in Windows or the Dock in Mac OS offer attentive memory support for applications and documents. Browsers also use tabs. None of these are super. Research has looked at how to improve these, such as approaches offering spatial modality: Scalable Fabric, more structure: GroupBar, and temporal and visual modalities: WindowScape, RelAltTab.

Most of these designs were devised before we had a deeper understanding of the brain. How can we rethink some of our design decisions with a better understanding of the brain and its support for memory? Our understanding of attentive memory is still growing, but one thing we can start asking ourselves is: What are the attentive memory failures that a user may face, and what are the interface elements that I can provide to support them.

Why are we still using ideas about memory from the 60′s to build programming tools?

As with the beginning of most fields of research,
discovery is born from stumbles in the dark.
Then beacons appear–theoretical stakes laid by early pioneers,
a way for researchers to make sure-footed progress.
But we were not meant to settle here.

Up to now, we have been building program tools based on conceptual models decades old,
founded on psychology research that is even older. As one of the authors of
a prominent conceptual model recently stated, “These models are being used long-past their shelf-life”.

For the programmer, almost no tool exceeds that of the brain and its ability for memory.
Despite this simple observation, almost no programming tool is built based on our understanding of the brain and its ability for memory. Our conceptual models and theories still hold outdated views on the brain and memory.

In my research, I building conceptual models for supporting programming tasks
based on a theory that understanding the cognitive neuroscience of human memory is essential for understanding how to build tools that support programmers. In doing so, I abstract out some key concepts and principles about human memory from modern neuroscience literature, in a manner that (hopefully) allows researchers and toolsmiths to build better programming tools.

In no uncertain terms, programmers need our help. Despite human memory’s remarkable ability, memory stressors are chipping away at programmer’s productivity. Interruption devastates memory and continues to make tasks twice as long to perform, and have twice as many errors [Czerwinski 04]. In a study of such interruptions, Parnin and Rugaber [Parnin 10] analyzed interaction logs of 10,000 programming sessions from 86 programmers and found that in a typical day, developers rarely are able to program in long continuous sessions. Instead, a developer’s day is fragmented into many short sessions (15-30 minutes) interspersed with occasional longer ones (1-2 hours). Further, at the start of each of the longer sessions, a programmer often spends a significant amount of time (15-30 minutes) reconstructing working context before resuming coding.

Age is also becoming another relevant stressor on memory; as demyelination [Andrews-Hanna 07] begins just a few years completing a college degree, followed by the specter of ageism. As Stephenson put, Software development, like professional sports, has a way of making thirty-year-old men feel decrepit [Snow Crash]. Take a stroll down a google or EA office, teeming with thousands of 20-somethings, and handfuls of “ancient ones”.

In the next series of blog posts, I’ll talk about 5 types of human memory, and then some ideas for supporting them in our programming tools.

Part 1 – Attentive Memory.

“

UPDATE:

Commonly, we need to perform a programming task that we do not know how to do or is not well-documented. Collectively, we perform these reoccurring programming tasks all the time, but only a few of us actively blog or contribute to sites like stackoverflow with answers. Thus, there are many remaining programming tasks, which are never documented for other teammates (on internal company blogs), don’t have any (good) official documentation, or discussion on blogs or Q&A sites.

If we can make it easier for people to blog, or share these experiences, then collectively we can make learning about these things a whole lot easier. This might also make other situations, like a task-hand off to another teammate easier. This post is a research demonstration of a technique that analyzes the coding history of a programmer and then automatically creates a blog summary of how they programmed that bit of functionality. This is still at the earliest stages, but I hope there is a lot of room for growth and expansion.

Check out the demonstration, and then some of the next steps below.

Proof-of-concept: Auto-generating a coding task blog post

This is an semi-automatically generated blog post created with the aid of some research tools I’m developing!

First, I’ll show you how this post got here, and then use it to illustrate a recent coding task involving programmatically “publishing” a blog post from C# code.

Auto-generating code snippets in Visual Studio.

First, click “Auto-Populate” (by default looks at most recent activity in 24 hours).

This populates the code narrative with a series of code changes, chunked by files, and ordered by median change activity per file.

The code snippet has symbolic links such as method definitions that can be clicked to return to that location in code (if more context is needed). The code snippet can be edited or removed as needed.

Another useful item is a section. Sections can be used to break up the code snippets, and can even be expanded or collapsed to ease review and edit of the code narrative.

Finally, explanatory text or an exposition item, such as this paragraph, can be inserted, to help add clarification to a code snippet. Any item can be dragged and dropped to reorder items to the desired position.

Sample Coding Task – Programmatically publishing a blog post with C#

First, I added a Publish link to my Code Narrative tool window in Visual Studio. We’ll eventually hook up the command in the view model below.

                <Label>
                    <Hyperlink Command="{Binding PublishCommand}">Publish</Hyperlink>
                 </Label>

Next, I create some simple data structures for the published blog post and site.

    public class BlogPost
    {
        public string Title { get; set; }
        public DateTime DateTime { get; set; }
        public List<string> Entries { get; set; }
    }

    public class BlogSite
    {
        public string Url = "http://blog.ninlabs.com";
        public string User = "******";
        public string Password = "*****";
    }

Next up, I create a BlogConnector which will provide the primary abstraction for making a connection to a blog site.

    class BlogConnector
    {
        private MetaWeblogClientProtocol protocol;
        BlogSite m_site;
        public BlogConnector(BlogSite site)
        {
            m_site = site;
            var url = site.Url.TrimEnd('/');
            protocol = new MetaWeblogClientProtocol();
            protocol.Url = url + "/xmlrpc.php";
            protocol.Credentials = new NetworkCredential(site.User, site.Password);
        }

        public string NewPost(BlogPost content, bool publish)
        {
            // Convert XML RPC post
            var post = new Post();
            post.title = content.Title;
            post.dateCreated = content.DateTime;
            post.description = string.Join("\n", content.Entries);
            post.categories = new string[]{"tutorial", "auto-blog"};

            // first paramter, blogId, is ignored by wordpress.
            return protocol.newPost("", m_site.User, m_site.Password, post, publish);
        }
    }

But, MetaWeblogClientProtocol is where all the magic happens. This is using the XML-RPC protocol to do the actual interaction with the wordpress backend.
This requires a 3rd party library, which can be found here.

    // minimal wordpress client (new post only), using the MetaWeblog API.
    // based on metaweblog xml rpc client at:
    // http://msdn2.microsoft.com/en-us/library/bb259701.aspx

    // blog post information.
    [XmlRpcMissingMapping(MappingAction.Ignore)]
    struct Post
    {
        public DateTime dateCreated;
        public string description;
        public string title;
        public string[] categories;
    }

    internal class MetaWeblogClientProtocol : XmlRpcClientProtocol
    {
        // new post.
        [XmlRpcMethod("metaWeblog.newPost")]
        public string newPost(
        string blogId,
        string userName,
        string password,
        Post content,
        bool publish)
        {
            return (string)this.Invoke("newPost", new object[] { blogId, userName, password, content, publish });
        }
    }

Next, we translate my “narrative” data structures to HTML equivalents. The code tag is targeting a wordpress syntax highlighting extension.

    public class StoryHtml
    {
        public List<string> FormatItems(IEnumerable<NarrativeItem> items,string lang)
        {
            var list = new List<string>();
            foreach (var item in items)
            {
                if (item.Kind == NarrativeItemKind.CodeSnippetBuilder)
                    list.Add(FormatCodeSnippet(item, lang));
                if (item.Kind == NarrativeItemKind.Exposition)
                    list.Add(FormatExposition(item));
                if (item.Kind == NarrativeItemKind.SectionBuilder)
                    list.Add(FormatSection(item));
            }
            return list;
        }

        public string FormatSection(NarrativeItem item)
        {
            if (item.Kind != NarrativeItemKind.SectionBuilder)
            {
                throw new ArgumentException();
            }
            return string.Format("<h2>{0}</h2>", item.SnippetText);
        }

        public string FormatCodeSnippet(NarrativeItem item, string lang)
        {
            if (item.Kind != NarrativeItemKind.CodeSnippetBuilder)
            {
                throw new ArgumentException();
            }
            // Actually, code is surrounded with [], but need to figure out how escape that!
            return string.Format("code lang=\"{0}\"{1}/code", lang, item.SnippetText);
        }

        public string FormatExposition(NarrativeItem item)
        {
            if (item.Kind != NarrativeItemKind.Exposition)
            {
                throw new ArgumentException();
            }
            return string.Format("<p>{0}</p>", item.SnippetText);
        }
    }

Finally, we can hook up the command, put everything together, and then publish this coding task!

                PublishCommand = new RelayCommand(
                () =>
                {
                    var blogConnector = new BlogConnector(new BlogSite());
                    var blogpost = new BlogPost()
                    {
                        Title = "Auto-blogging - Publishing a coding task to wordpress.",
                        DateTime = DateTime.Now,
                        Entries = new StoryHtml().FormatItems( Items, "csharp" )
                    };
                    blogConnector.NewPost(blogpost, false);
                },
                () => { return true; });

Next steps…

Coming up, hoping to expand this in several ways, and answer many research questions I have!

First a shout out to a previous post that has helped point out some of the basic technology approaches for doing this.

• working on algorithms that chunk up the code snippets more intelligently,
• auto-generating sections based on coding activity and breaks,
• auto-generating exposition based on activities and using as a place for adding interesting facts about the code snippet or task (copied X code from this blog post), or a troublesome exception that was happening,
• improving code narrative edit UI, and
• inserting images in Visual Studio (doing that manually now).
• Automatically extract a compilable and executable snapshot of the code needed for the task which can be included as a download.

Tabs were introduced in Visual Studio 2003 (although addins existed for Visual Studio 6). This is how they look now in Visual Studio 2010:

My best guess is that developer’s have a love/hate relationship with tabs. They’re great when they work, but many times they get cluttered, and developers have to go nuclear on them (“Close All Tabs”).

A simple fact is that many times developers simply forget what the heck was in the tab. Research backs this up: Ko et. al found developers relied on environmental cues such as tab placement and scroll bar positions in documents to know where anything is. In work looking at recorded navigation history of developers, commonly there will be patterns where developers will frantically switch around documents (“navigational jitter”) as they’re trying to find the right tab.

A second problem with development is context explosion. A developer’s task context typically involves a few files, but will quickly explode when exploring, debugging or searching through files. Above is a snapshot of one developer’s session measuring the number of actions (edits,clicks,etc) taking on a file before switching. You may notice that the developer focuses on one element for a while, before jumping around several locations until settling in the next major location of focus.

The napkin idea

What if we did two things. First, let’s put a preview of the content in the tab. Memory research suggests that “names” are poor cues for recall. Second, let’s associate the tab with the major action it was involved with.

Update (New Screenshot):

Below the name is a small document preview. Thea action icons indicate how the document was used. “Search”, “Debug”, “Edited” icons let you know what you did with a tab and being able to perform grouping and pruning operations, such as closing all tabs opened in a search.

This can go in many directions. Tabs, love them or hate them, can really use a make-over.

Continuing my last post on creating a basic history extension, let’s actually do something! Here, we dive into capturing events to code documents in Visual Studio and using that to create a help build a local code history for edits.

First, let’s make a new SaveListener class that will subscribe to Running Document Table events.

class SaveListener : IVsRunningDocTableEvents3 {

We need to request the SVsRunningDocumentTable service and subscribe to it’s events.

IVsRunningDocumentTable m_RDT;
uint m_rdtCookie = 0;

public bool Register()
{
   // Register events for running document table.
   m_RDT = (IVsRunningDocumentTable)Package.GetGlobalService(typeof(SVsRunningDocumentTable));
   m_RDT.AdviseRunningDocTableEvents(this, out m_rdtCookie);

   return true;
}

Because SaveListener implements the IVsRunningDocTableEvents3 interface, we’ll able to pass this to receive event notifications.

For this blog post, let’s try to grab and cache a copy of files about to be saved. This snippet will let us get some information about the file about to be saved such as the file name:

public int OnBeforeSave(uint docCookie)
{
   uint flags, readlocks, editlocks;
   string name; IVsHierarchy hier;
   uint itemid; IntPtr docData;
   m_RDT.GetDocumentInfo(docCookie, out flags, out readlocks, out editlocks, out name, out hier, out itemid, out docData);

   CopyFileToCache(name);

   return VSConstants.S_OK;
}

Here is some C# IO code that will take care of copying the file to a cache directory:

public void CopyFileToCache(string file)
{
   try
   {
      var newPath = PrepareDocumentCache(file, DateTime.Now);
      System.IO.File.Copy(file, newPath, true);
   }
   catch (Exception ex)
   {
     Trace.WriteLine(ex.Message);
   }
}

private string PrepareDocumentCache(string file, DateTime day)
{
   string relativePath = file.Replace(SolutionBaseDirectory + "\\", "");
   var dayStr = MakeDayString(day);
   var time = MakeTimeString(day);
   var newPath = System.IO.Path.Combine(ContextRepository, dayStr, relativePath + "$" + time);
   var dirPath = System.IO.Path.GetDirectoryName(newPath);
   if (!System.IO.File.Exists(dirPath))
   {
      // This will create subdirectories too if missing...
      System.IO.Directory.CreateDirectory(dirPath);
   }
   return newPath;
}

private static string MakeDayString(DateTime day)
{
   return string.Format("{0:0000}", day.Year) + "\\" + string.Format("{0:00}", day.Month) + "\\" + string.Format("{0:00}", day.Day);
}

private static string MakeTimeString(DateTime time)
{
   return time.ToString("hh.mm.ss.fff.tt");
}

Let’s add some code to shutdown the save listener.

public void Shutdown()
{
   if (m_RDT != null)
   {
      m_RDT.UnadviseRunningDocTableEvents(m_rdtCookie);
      m_RDT = null;
   }
}

Finally, let’s hook up the save listener from our Package we created in the first post so we can start it up and shut it down at the right time:

SaveListener m_saveListener;
public int OnAfterOpenSolution(object pUnkReserved, int fNewSolution)
{
   m_dte = (EnvDTE.DTE)this.GetService(typeof(EnvDTE.DTE));
   if (m_dte == null)
      ErrorHandler.ThrowOnFailure(1);
   if( m_dte.Solution != null )
   {
      Trace.WriteLine(m_dte.Solution.FullName);
      m_saveListener = new SaveListener();
      m_saveListener.SolutionBaseDirectory = System.IO.Path.GetDirectoryName(m_dte.Solution.FullName);
      m_saveListener.ContextRepository = ".codeHistory";
      m_saveListener.Register();
   }
   return VSConstants.S_OK;
}

Ok! That’s the basic idea behind creating a local history extension. Here, we’re just copying the file to a cache. Eclipse does something similar, except they use a heap directory structure. I even have having each save be a commit to git, and it works nicely!

For some applications of a local history repository, check out my earlier post of “code diffs redesigned”. Another idea is to try out some developer analytics. What API calls was I spending all my time mucking around with? Maybe there is a lesson to be learned there?

I will publishing a series of blog posts describing how to extend Visual Studio (VSX in particular) to maintain a history of code changes or actions within the IDE. The audience for these articles would most likely be researchers who are trying to understand how to do interesting developer analytics, intelligent work summaries, or build interesting work history visualizations, or next-gen developer tools.

I will be starting out with how to set up an package which will be the core entry point into Visual Studio receiving events. It used to be prior to Visual Studio 10, you could only create a VSIP Package that allowed deep integration with Visual Studio if you jumped through all sorts of hoops and hurtles.

First, make sure you have the Visual Studio SDK SP1 installed. These will install some private assemblies and create some project templates.

Let’s create the project called VSHistory.

This will create a new file VSHistoryPackage.cs. One attribute that we will want to add that is essential for a history extension is an “autoload” attribute. This will make sure the package is loaded and initialized without having to launch it from a menu: in this case, it is autoloaded when an solution is available.

[ProvideAutoLoad(VSConstants.UICONTEXT.SolutionExists_string)]

Next thing we will want to add is an IVsSolutionEvents interface. This will make sure we can respond to events like a project or opening.

public sealed class VSHistoryPackage : Package, IVsSolutionEvents

   public int OnAfterOpenSolution(object pUnkReserved, int fNewSolution)
   {
      return VSConstants.S_OK;
   }

Ok. Let’s start by with learning an important concept as well as grabbing a handy service.
The EnvDTE automation framework for Addins is old, obsolete, and a second-class citizen, but sometimes there are some things only it provides. You can ask for this service using this pattern:

   EnvDTE.DTE m_dte;
   public int OnAfterOpenSolution(object pUnkReserved, int fNewSolution)
   {
      m_dte = (EnvDTE.DTE)this.GetService(typeof(EnvDTE.DTE));
      if (m_dte == null)
         ErrorHandler.ThrowOnFailure(1);
      if (m_dte.Solution != null)
      {
         Trace.WriteLine(m_dte.Solution.FullName);
      }
      return VSConstants.S_OK;
   }

To start listening to these events we need to get the Solution service and listen to it’s events.

uint m_solutionCookie = 0;
protected override void Initialize()
{
   Trace.WriteLine (string.Format(CultureInfo.CurrentCulture, "Entering Initialize() of: {0}", this.ToString()));
   base.Initialize();

   IVsSolution solution = (IVsSolution)GetService(typeof(SVsSolution));
   ErrorHandler.ThrowOnFailure(solution.AdviseSolutionEvents(this, out m_solutionCookie));
}

Next post will cover using the running documents table to capture saves to code documents.

The International Conference on Program Comprehension (ICPC 2011) was a charming affair, bringing together a diverse set of researchers from around the globe.

Here are some of my personal reflections:

Three Research Perspectives

Leon Moonen drew upon centuries of wisdom collected by architects and map makers to find design principles of wayfinding that could be applied toward software exploration and navigation interfaces.

Margaret Burnett showed how studying the barriers faced by a segment of a population (in this case gender) could be used to for making design improvements that benefit the whole population.

Margaret-Anne Storey reflected on the trials and tribulations of trying to bring a software visualization system (SHriMP) to become loved and adopted by professional programmers. In the process, she emphasized on the importance of working with cognitive theories, but also pointed out that our current set of program comprehension theories have long since expired.

Technical Program

The technical program was both interesting and strong with a nice mix of topics. Which was something, personally, I did not see at ICSE.

The program started out on a technical note: clustering, identifier splitting, smoothing filters to improve information retrieval (IR) tasks in software. The tradition of awarding best papers to IR papers also continued with “Improving IR-based Traceability Recovery Using Smoothing Filters” taking the price.

Another strong topic that emerged was the study of the interaction or work history of developers. Annie Ying looked at edit patterns and their relation of task difficulty. David Röthlisberger studying patterns of artifact relevancy for different types of programming tasks. Lile Hattori had an well-performed study studying the benefit of replaying code changes in the IDE in comparison to viewing differences in subversion.

There were also some really nice empirical studies and tools. Anja Guzzi gave a spunky presentation (one of the best talks) on collaborative bookmarks in the IDE (Check out Pollicino). Stefan Endrikat laid a critical eye over aspect-oriented programming (AOP) with a study that cast doubt over the purported benefits of AOP.
Daqing Hou manually poured over newsgroup postings to get a better understanding of why developers have trouble about certain API methods.

Industrial Challenge

The industrial challenge was a different but refreshing way of engaging researchers. There were two parts: first use current code comprehension tools to try find and fix three bugs in a robot controller part. Second, use those insights to write fictional emails to other stakeholders: customer support, another company tech lead, and CEO.

There were several interesting outcomes:

- No submission used a IR-based technique despite popularity in community.
- Two submissions used statistical debugging techniques, and both performed very poorly due to superficial understanding of the bugs.
- The simplest techniques worked the best: program slicing and code differences.
- The social aspect (emails) proved to be the most difficult aspect for participants. The challenge exposed the need to explore more of the ecosystem of program comprehension, and how different stakeholders may have different information needs that are not currently targeted by any research effort or tool.

See how the winner did it: part 1, part 2, and final thoughts.

Other Notes

- We need to archive conference tweets. They’re already gone from twitter… UPDATE: Andrian Kuhn has archived them here.

- Many researchers don’t have their slides, papers, or tools online!

- I would like to see more IR-papers that focus on tool-building and describe the experiences of putting these tools in the hands of developers.

Hunting Particles

Tasks, for the purpose of this post, are assigned units of work. In practice, this may be a “story” or “defect” from an iteration in the agile world, or a “work item” in TFS land, or even a “feature” in trackers such as trac. There are many names given to tasks, but for the programmer, ultimately they are externally assigned responsibilities.

Tasks are also broken down. For example, in version one, a “epic” is often broken down into several “stories”, and ending with “tasks” that may be split across multiple iterations or developers.

But does it end there? Are tasks indivisible atoms of software development? Survey says, not likely. When 366 professional developers were asked about the nature of assigned tasks, more often then not, developers felt the actual work had to be broken down much further, usually after exploration and experimentation and involved elaboration on many steps, issues, and TODOs.

Task-focused Interfaces

Although we have excellent tools for issue and task tracking, there is a gap in tool support for managing and tracking the knowledge and breakdowns beyond the initial assigned responsibility. Instead, these sorts of information are scattered on scraps of paper, white boards, source code comments, or human memory. Studies on interruptions of programmers show that after an interruption, programmers spend 15-30 minutes gathering loss information before making their first edit.

The most successful attempt at solving this problem has been, Mylyn, a “task-focused interface” available in Eclipse that will track the “task context”.

There are two weaknesses that can be improved upon:
1) The idea of “task context” has been mostly limited to the files and methods that were visited or edited during a particular task. Surely there is much more bits of “context” than places in code.
2) Breakdowns of tasks are a little heavy weight. Although the idea of “subtasks” are supported, they are not frequently used in practice. Eclipse usage data suggests that less than 10 users have created subtasks compared to the hundreds that have “activated” Mylyn tasks.

Task-focused Interfaces 2.0: Tasklets and Worklets

I have been researching alternatives how IDE interfaces can better support task information. I have looked at spatial interfaces, such as CodePad, that can serve as an “external thought-space”. Code Bubbles is also an interesting stab at this concept.

More recently, I’ve been looking at how to support more capturing “particles” of a task. Tasklets, can be opened as tabs, and let you keep a separate workspace and state for different activities (information seeking task without trashing your context). Worklets record everything about programming state. Stack traces, search results, code diffs, code fragments and annotations, prospective reminders, thoughts, issues, and objectives. These all exist within a tasklet or task’s workspace. Worklets also include automatically associated properties, like url provenance when copying code from a stackoverflow post.

All these are stored in an local history repository. But they can be pushed and promoted beyond the “personal” boundary and into “public” or “team” space. Sorta sounds like git, but for your context. They are also fully searchable. Rather than just searching code, you can search for a “recent” change, or information held in a “recent” stack trace.

More to come…

This is my second attempt at a software-related comic, but this time it is completely hand drawn.

This is loosely based on the “epic battle” that took place between the designers of the Java language when deciding on language features during the early days of its formation.

Click on the picture for full size:

VSX extensions/packages published and installed through the visual studio gallery will automatically check for updates and allow you to install the newest versions.

If you’re not deploying your extension through the gallery, or just want auto-update capability for yourself, you can mimic this process yourself using the IVSExtensionManager interface. But when I wanted implement this, exactly how to do it was not documented anywhere.

The process is actually straight-forward:

You’ll need to reference:
Microsoft.VisualStudio.ExtensionManager.dll
Microsoft.VisualStudio.OLE.Interop
Microsoft.VisualStudio.Shell.Interop.8.0

Warning, the extension assembly isn’t published in the .NET components, you’ll have to manually browse to the GAC to reference it:
C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.VisualStudio.ExtensionManager\v4.0_10.0.0.0__b03f5f7f11d50a3a\Microsoft.VisualStudio.ExtensionManager.dll

var manager = Package.GetGlobalService(typeof(SVsExtensionManager)) as IVsExtensionManager;

Using the manager, get a list of installed extensions

var names = new string[] { "Ganji", "Ganji.History", "Ganji.Margin" };
var myExtensions = manager.GetInstalledExtensions().Where(ext =>names.Contains(ext.Header.Name)).ToList();

Next, we want to be able to download and then install a new version of our extension.
This is the general logic. The details follow.

                bool needsRestart = false;
                foreach (var extension in myExtensions)
                {
                    var newVersion = FetchIfUpdated(extension, webRepository.Where(r => r.Name == extension.Header.Name).SingleOrDefault());
                    if (newVersion != null)
                    {
                        Install(manager, extension, newVersion);
                        needsRestart = true;
                    }
                }
                // Client needs to restart
                return needsRestart;

You can check and fetch a new version with the IVsExtensionRepository manager.

        private IInstallableExtension FetchIfUpdated( IInstalledExtension extension, RepositoryEntry entry)
        {
            var version = extension.Header.Version;
            var repoManager = Package.GetGlobalService(typeof(SVsExtensionRepository)) as IVsExtensionRepository;
            try
            {
                var newestVersion = repoManager.Download(entry);
                if (newestVersion.Header.Version > extension.Header.Version)
                {
                    return newestVersion;
                }
            }
            catch (Exception ex)
            {
                // may not have internet connection, etc...
                Console.WriteLine(ex.Message);
            }
            return null;
        }

But you have to define your own IRepositoryEntry, which will be used as a parameter in the Download call.

 class RepositoryEntry : IRepositoryEntry
    {
        public string Name { get; set; }
        public string DownloadUrl { get; set; }
        public string VsixReferences { get; set; }
    }

Finally, installing the extension involves disabling your old extension, uninstalling it, installing the new extension, then enabling it. A restart will be required before it takes affect however.

        private RestartReason Install( IVsExtensionManager manager, IInstalledExtension currentExtention, IInstallableExtension updatedExtension)
        {
            // Uninstall old
            manager.Disable(currentExtention);
            manager.Uninstall(currentExtention);

            // Install newer version
            var restartReason = manager.Install(updatedExtension, false);

            // Check newly installed version and enable (which is not the case by default)
            var newlyInstalledVersion = manager.GetInstalledExtension(updatedExtension.Header.Identifier);
            if (newlyInstalledVersion != null)
            {
                manager.Enable(newlyInstalledVersion);

                Console.WriteLine(string.Format("Updated: from {0} to {1} at {2}",
                                currentExtention.Header.Version,
                                newlyInstalledVersion.Header.Version,
                                newlyInstalledVersion.InstallPath));
            }

            return restartReason;
        }