Use of text in the alt attribute of images is widely misinterpreted and not always correctly implemented. It is possible to be too accessible with this attribute, overly describing peripheral content or added repetition to the page. 

So how do you know when, and what, should be added to the alt attribute? The first place to call upon is the Web Content Accessibility Guidelines (WCAG) 2.0 and what they say on non-text content:

WCAG 2.0 1.1.1 Non-text Content

1.1.1 Non-text Content: All non-text content that is presented to the user has a text alternative that serves the equivalent purpose, except for the situations listed below. (Level A)

Controls, Input: If non-text content is a control or accepts user input, then it has a name that describes its purpose. (Refer to Guideline 4.1 for additional requirements for controls and content that accepts user input.)

Time-Based Media: If non-text content is time-based media, then text alternatives at least provide descriptive identification of the non-text content. (Refer to Guideline 1.2 for additional requirements for media.)

Test: If non-text content is a test or exercise that would be invalid if presented in text, then text alternatives at least provide descriptive identification of the non-text content.

Sensory: If non-text content is primarily intended to create a specific sensory experience, then text alternatives at least provide descriptive identification of the non-text content.

CAPTCHA: If the purpose of non-text content is to confirm that content is being accessed by a person rather than a computer, then text alternatives that identify and describe the purpose of the non-text content are provided, and alternative forms of CAPTCHA using output modes for different types of sensory perception are provided to accommodate different disabilities.

Decoration, Formatting, Invisible: If non-text content is pure decoration, is used only for visual formatting, or is not presented to users, then it is implemented in a way that it can be ignored by assistive technology.

While points 1 to 5 are fairly explanatory, it's worth expanding on the fifth point and what is meant by a decorative image. The guideline's footnote for pure decoration states 'serving only an aesthetic purpose, providing no information, and having no functionality'. The current version of the HTML5 spec (as of 16 November 2010) expands on the requirements for providing text to act as an alternative for images in the section A purely decorative image that doesn't add any information:

In general, if an image is decorative but isn't especially page-specific, for example an image that forms part of a site-wide design scheme, the image should be specified in the site's CSS, not in the markup of the document.

However, a decorative image that isn't discussed by the surrounding text but still has some relevance can be included in a page using the img element. Such images are decorative, but still form part of the content. In these cases, the alt attribute must be present but its value must be the empty string.

When is an image decorative?

Border swirls, images for rounded corners and  background gradient images are all obvious examples. Where the line blurs is with photos and illustrations. The example given in the spec refers to an image that is purely decorative despite being relevant to the content. 

In the above example the park photo is relevant to the blog post but is purely decorative. This can, however, change dependent on the context of the copy around it. 

In this example the image is referred to directly in the text, so it is important to be able to understand what is in the picture. 

Delving further into some of the W3C's editors drafts on HTML5 the Techniques for providing useful text alternatives is a very thorough and descriptive document that provides many useful examples. In it one of the examples, perhaps unwisely, offers two correct but conflicting viewpoints. 

As the image bears no direct relation to the content of the page it is considered appropriate to use an empty alt attribute. It can also be considered appropriate to provide a brief description of the image as some users who cannot view images appreciate having information provided about images of paintings and photographs regardless of the context in which the images are used. As decisions about when to provide a text alternative are based on context of use, both options are considered to be conforming HTML5.

This inevitably leads to the divisive grey area of the W3 guidelines - interpretation and developers opinion. Context is key when defining how to use the alt attribute. The alternative text should not be redundant or repeat content already on the page.

In this example the title of the article is Theoretical Physics, so it's relevant to have Albert Einstein in the alt attribute. Note that this isn't overly described as 'photo of Albert Einstein' or 'Albert Einstein, Theoretical Physicist' as the reader can clearly make the connection that it's either a photo or that Einstein was a theoretical physicist as that is explained in the context of the article. 

In this example the alt attribute is left empty as the heading and subsequent content provides the context for the image. Adding 'Albert Einstein' would be a repetition of the heading text. 

How do I interpret the WCAG 2.0

The WCAG 2.0 is not a concise list of the right way and wrong way to implement web content. A large majority of the points are open to interpretation and in these cases, as long as the point of view is backed up clearly, the developers decision should be final. Everyone should be open to suggestion as discussion is a great way to encourage and debate individuals points of view. 

In this article I referred to full released W3C specs as well as several editors drafts. Whilst some of the content of the drafts may change, the base idea or reasoning is likely to remain constant. However, I do suggest you do your won research and make up your own mind.