Wednesday 26 March 2014

Tutorial 13

Layouts...

pie layout

>var pie = d3.layout.pie()
. value(function (d) {return d;})

Watching this but it's really at the edge of my knowledge. - arghh...

Get some data and try making some graphs!!

Donut chart and pie chart use the same code.

Some key concepts:

Getting the data in
Creating colour scales
Creating an SVG canvas
Using a d3 path generator to create the svg code.
binding data to the path elements (objects).
adding text to the graphic elements.

With this basic code, we can create pie and donut charts.

So how many Schools with Bronze, Silver, applying etc...

What kind of data do I want to put in a pie chart?

Let's use the Athena SWAN data for some visualisations.

While not scientific, it still feels like a good place to start.

Tuesday 25 March 2014

Path component - video 11

Path component in SVG

Can be used to create any shape!

d3 has path generators.

Stored in the "d attribute" - I don't understand that yet.

draw a line using data...

Debugging d3

Go to console and you get error messages there.

Lesson 12
Making an arc - create a path.
var arc = d3.svg.arc()
.innerRadius
.outerRadius
.startAngle
.endAngle

Create a path

Angles NOT measured in degrees - rather we use Radians.
When a portion of the circumferance equals the radius then that equals one radian.

Number of radians in a circle is PI multiplied by 2.

D3 video tutorial 10 - loading external data...

Data not normally hard coded.

Load external files.

Stored in the same folder as the index.html

Fetching data:
depends on the format:
json format -
d3.json("mydata.json", function (data) {code}

First argument is the path to the file.
Second argument - what must happen once the data is loaded.
- this will be a function - "call back function"
- all the code that depends on the data should be stored here!

Some good information here.

How to add text to our bar chart is covered and merits some revisiting.

Playing with Arrays in D3 - Youtube lesson 9

Vienno

Some of these are general javascript functions rather than D3 functions.

data.shift();
- Give you the first value in the array "data"

Can add items to the array
Probably .append

Remove items from the array

Sort the array
data.sort(d3.decending);

Find max of the array
d3.max(data);

Find min of the array
d3.min(data);

d3.extend(data);
Returns the min and the maximum.

d3.sum(data);
Add up all the elements.

Some statistical functions
d3.mean(data)
d3.median(data)

Lots more in the API reference

One last:
d3.shuffle(data);
moves them in a random order....

Monday 24 March 2014

My first D3 code for Athena SWAN Awards...

I wrote the HTML in Word. Saved it in text format with .html after the file.

The code is below.

I don't understand it and we are lacking BOTH the Y axis and X axis. It's not scaled which is unfortunate but hey I am making progress.

Can I improve this and serve it to the net?
I would REALLY like to add the colours of the Awards as a kind of blocked bar chart.

More to do...

Here's the code:

<!doctype html>

<html>

<head>

<title> Athena SWAN Awards in a D3 graph</title>

</head>

<body>

<h2> This graph shows the number of Athena SWAN awards across the last few years </h2>

var yrs = [2009, 2010, 2011, 2012];

var aws = [35, 30, 39, 93];

var width = 600;

var height = 500;

var canvas = d3.select("body")

.append("svg")

.attr("width", width)

.attr("height", height)

.append(“g”)

.attr(“transform”, “translate(100,0)”);

var bars = canvas.selectAll("rect")

.data(aws)

.enter()

.append("rect" )

.attr(“fill”, “red”)

.attr("width", function(d) {return (d*5);})

.attr("height", 50)

.attr("y", function(d, i) {return i *55});

</script>

</body>

</html>

D3 Video 4 - Binding data...

Keep the canvas element

create data

var dataArray = [20, 40, 50]

visualise this with a simple bar chart

var bars = canvas.selectAll("rect")
.data(dataArray)
.enter()
.append("rect")
.attr("width", function(d) (return d;))
.attr("height", 50)
.attr("y", function(d, i) (return i*100))

So I just put this into Word and loaded it into Google Chrome and it made the bars.

Great!!

Very happy.

I need to commend the video maker.

D3 video 3

Adding style elements - using CSS

style method
.style
two arguements

so to turn a paragraph red:
d3.select("body)
.apend("p")
.style("color", "red")
.text("hi");

</script>

CREATE SVG shapes

use the .attr method

Create a container on the web page first
d3.select("body")
.append("svg")
.attr("width", 500)
.attr("height", 500);

worth creating a variable - GOOD practice

var canvas = d3.select("body")

.append("svg")

.attr("width", 500)

.attr("height", 500);

make a circle

var circle = canvas.append("circle")

.attr("cx", 250) - cx gives a position

.attr("cy", 250) - cy gives a position

.attr("r", 50) - r is radius

.attr("fill", "red") - gives a color to the circle.

This is useful and I think I need to give it a go somewhere.

Where can I try my d3 script?????

Local file will work!!!

Watching d3 videos on Youtube...

A nice break from books and websites just for a change I have checked out some Youtube videos.

The are made by d3Vienno and have quite a nice style.

I have just looked at the second video and learned the following:

Looking at the d3 code:

Some really core concepts:

D3 object

Functions or methods

Arguments

Method chaining – methods separated by dots.

d3.select(“body”).append(“p”).text(“Hi, what’s up?”);

</script>

d3 is the object - Calling the d3 object

select, append, text are all methods or functions.

Brackets denote the arguments.

Another function

Append – so adding something to the body – in this case paragraph – this is an argument

Then we have the text method specifying the

method and object separated by a dot.

This is method chaining.

Connect selections and the methods we want to perform on that.

Basic methods in d3

select – selecting elements

append – add elements

A comment about the consol – in Chrome.

Getting D3 into wordpress...

It's going to take a bit of trial and error but here is a tutorial from someone committed to the process.
Going to have to add it to my list of todos for the moment.
Gotta do something else :-)

Learning about D3...

Using web books, website and most recently youtube, I have been building my knowledge of D3 - Data-Driven Documents.

It's a Javascript Library that allows the visualisation of data on a web page.

Steep learning curve but it will allow the generation of advanced visualisations.

It's a data presentation method not a data exploration method.

For data exploration, R is still the best. ggplot comes up again and again as a good way to go forward with that.

Requires some knowledge of:

HTML
CSS
DOM - Document Object Model - hierarchial structure of the a web page.

I don't really understand this concept yet but I'm getting there!

Need to know how to edit and deliver a web page as well.

Need a text editor for editing web pages - these seem integral to some browsers.

I think I probably will want to get another computer for myself too!

Long list of methods and objects in the documentation.

I think I need to try making a graph so that I can really make some of the learning a bit more concrete!

Found another resource:

https://leanpub.com/D3-Tips-and-Tricks/read

Another interesting thing:

There would seem to be a D3.js Wordpress plugin - this sounds like good news.

Can I make it work?

Friday 21 March 2014

Making a Bar Chart I

So, the html is here

http://bl.ocks.org/mbostock/7322386

It works when I load it into word, save it as text only and open it in the browser.

That's good.

It looks quite simple but do I understand it?

D3 on GitHub

Shamelessly copy and pasted:

"D3.js is a JavaScript library for manipulating documents based on data. D3 helps you bring data to life using HTML, SVG and CSS. D3’s emphasis on web standards gives you the full capabilities of modern browsers without tying yourself to a proprietary framework, combining powerful visualization components and a data-driven approach to DOM manipulation."

I really like the ideas but I am not sure I understand the terms :-(

Do I really understand what a JavaScript library is?
Well in the context of D3.js, I guess it a set of JavaScripts that will do particular things. Somehow integrating HTML, SVG and CSS. Hopefully doing some of the work for us....

DOM

What is this:
Document Object Model (DOM)

Another D3 tutorial...

So here is another tutorial in D3.

Advice:
Just copy-paste this example in a file named "index.html" opened in your favorite editor and in a recent browser.

I have just done this using Word as my editor and it seems to work.
Great news!!!

I am guessing that it's the script in the head of the page that calls the javascript library from http://mbostock.github.com/d3/d3.js

It puts a nice circle on the page and when I put my mouse over the circle it changes colour.

This is a great start and represents a great step forward for me.
Small step but great!

:-)

Java Script and D3 again....

A key element of JavaScript is the script tag... i.e. the word script in html tags (less than and more than shapes. According to the Make a Bar Chart tutorial. You have to first load D3. However, I don't know how to do that!!! Arghhh!!!!

Java script...

It's also necessary to learn a bit of Javascript but this is a bit challenging.

My First JavaScript

Click the button to display the date.

This doesn't seem to work :-(
There is lots to learn....

Trying to understand D3 and the like....

So in an attempt to understand D3, I have been working through some of the information on this page D3 for Mere Mortals.

However, I can't really make the code work.

I have had to learn a bit about SVG - scalable vector graphics.

You can use svg to create code on websites to give you boxes and the like. Clearly boxes are just a simple start but hey... Can I add code to this page to make a box here with SVG? YES!!! This seems to work. Here is a different colour box: So the coding goes as follows svg is a html tag like heading or paragraph. There are various shapes, colours, lines and the like.
Looking at the code: you have "rect" for rectangle of particular width and height. You have fill, stroke, stroke-width, fill opacity and the like.
I got the code from w3schools.com.

Wednesday 19 March 2014

Heatmaps...

I am working through the exercises in the Flowing data book - pages 230 onwards.

I like the look of the heatmaps.

It requires a bit of data munging.
One key thing to do is to put the data into a matrix.
A data from will give an error message!

generally that's not too difficult but in a matrix the data type has to be uniform across all cells!

> ball_matrix <- data.matrix(bball)

That does it. Not too difficult.

Within heatmaps colours are important.

ColorBrewer can be worth installing.

> library (RColorBrewer)

>bball_heatmap <- heatmap(bball_matrix, Rowv=NA, Colv=NA, col = brewer.pal(9,"Blues"), scale="column", margins=c(5,10))

Then makes this:

I quite like the look of this but I am not sure about the quantitation of the colours.
I am a bit suspicious of this.

Still it's interesting.

I have also looked at Chernoff Faces but I am not convinced.
Start charts are mentioned but I don't really like them either.

Still making good quality Heatmaps is probably a good skills to have.

That's all for now.
I'm tired and I want to go to bed.
I might settle for meditation...

In the book there are spaces between the boxes in the heatmap but that's not the way they are coming out of R for me.
This is a bit puzzling.
I tried searching for a bit more info but so far not successful.
Mmmm....

Friday 14 March 2014

Bubble charts in R....

Working through the exercises on pages 194 to 200.

> symbols(crime$murder, crime$burglary, circles=radius, inches=0.35, fg="white", bg="red", xlab="Murder Rate", ylab="Burglary Rate")

add labels with the text command

> text(crime$murder, crime$burglary, crime$stat, cex=0.5)

This data set is from flowingdata.com and is quite useful.

> crime <- read.csv("http://datasets.flowingdata.com/crimeRatesByState2005.tsv", header=TRUE, sep="\t")

I need to make a basic list of what I need to be able to do with R.

This includes

correlations
multiple correlations
adding trend curves
reporting on the stats.
Kaplan Meier curves
Bubble plots
Strip plots
Word clouds (understand the stats)

I need my own cheat sheets.

I need to get some of my own data to manipulate.

Then I need to work out strategies (perhaps Illustrator - to make these plots look as good as possible.

Wednesday 12 March 2014

D3 and Javascript

D3 allows you to create "an interactive SVG bar chart with smooth transitions and interaction."
I want to be able to do that!

Let's see if I can work through a tutorial...

Start with:
'Let's Make a Bar Chart'

Mmm, assumes you know how to edit a web page - well maybe...
assumes you know how to include d3.js on the page - well no!

Going back to Javascript
http://www.w3schools.com/js
interesting command:
changing output:
document.write("Hello world")

buttons
<button type="button" onclick="alert('You fool')">Don't Click Me!</button>

getting buttons to change things:

<!DOCTYPE html>
<html>
<body>

<h1>My First JavaScript</h1>

<p id="demo">
JavaScript can change the content of an HTML element.
</p>

<script>
function myFunction()
{
x=document.getElementById("demo"); // Find the element
x.innerHTML="Hello JavaScript!"; // Change the content
}
</script>

<button type="button" onclick="myFunction()">Click Me!</button>

</body>
</html>

Key line here is
document.getElementById("some id"). This is defined in the HTML DOM.

changing images too.
Clever - I like it.

Question again - how do I put this in a web page. I need the image and the like.

I could create a website with wix.com to try out some Javascript...

Let's go back to D3 and have a little work through even if I can't understand everything...

Hmm, a bit of work to do here.
However, I would really like to master some of this stuff to get an interactive graphic on a web page....
Do I need to buy another book?
Perhaps one on D3?

Thursday 6 March 2014

More about treemaps...

Looking at the background for treemaps:
"During 1990, in response to the common problem of a filled hard disk, I became obsessed with the idea of producing a compact visualization of directory tree structures."

If I understand this concept correctly, it's about deeper sets.

It's possible that treemaps might be useful for comparing changing hierarchies.

More useful than a pie-chart because of the hierarchial relationships.

CD4, CD8
Then inside CD4 and CD8

Generate a treemap for each patient - perhaps not very useful

Generate an average treemap for normal, normal ratio and inverted ratio.

Clearly there is a challenge reflecting the variation but we might be able to do the means.

Or maybe not....

Interesting....

Learning a bit about Treemaps...

History of Treemaps
http://www.cs.umd.edu/hcil/treemap-history/index.shtml

Making them with R
uses the
Package - "portfolio"
May need to download it
library(portfolio)

May need to
install.packages("portfolio")

The command then is:
>map.market
It's a bit complex....

I am working through the exercise in the Flowing Data book about Visualising Proportions (pages 157-160).

There is a typo on the page:
From the data downloaded from here:
>posts <- read.csv("http://datasets.flowingdata.com/post-data.txt")

> map.market(id=posts$id, area=posts$views, group=posts$category, color=posts$comments, main="Flowingdata Map")

The "id" column gives each individual box, I think.

The area of the box comes from the number of views.

Each of the posts is groups by category.

The colours reflect the number of comments.

So we have four dimensions of data here which is quite nice.

This is an interesting method of visualisation.

I don't know how useful it is right now but it is certainly interesting.

Interpretation of this requires some understanding.

Wordclouds again....

> feedback <- Corpus(DirSource("wordcloud/"))
> inspect(feedback)

> feedback1<-tm_map(feedback,stripWhitespace)
> feedback1<-tm_map(feedback1, tolower)

> feedback1<-tm_map(feedback1, removeWords, stopwords("english"))

> wordcloud(feedback1)

> wordcloud(feedback1, scale=c(5,0.5), colors=brewer.pal(8,"Dark2"))

How to specify axis interval unit...

There may be another and perhaps an easier way to do this but here is some interesting text:

https://stat.ethz.ch/pipermail/r-help/2009-April/193847.html

Trying to make my boxplot look nicer...

Default gives this:

Some ideas here:
http://thinkdatavis.com/2013/06/05/how-to-make-nice-boxplots-with-r/

frame = "f"

boxplot(grades$Grade_Dig, grades$Grade_Pres, xaxt="n", outline=FALSE, col = "gray", lwd=3)

lwd controls size of lines

advice from here:
http://grokbase.com/t/r/r-help/097yxa4jh3/r-how-to-change-the-thickness-of-the-lines-of-the-boxplot-outliers

this is what I have right now:

> boxplot(grades$Grade_Dig, grades$Grade_Pres, xaxt="n", outline=FALSE, col = "gray", lwd=3, frame="false")

If I enlarge and take a screen shot, I get this...

Wednesday 5 March 2014

Mean, SD and T-tests

Reading:
http://cran.r-project.org/doc/manuals/R-intro.html

Short reference card:
http://cran.r-project.org/doc/contrib/Short-refcard.pdf

35 values

Need to slice the values as the N/A make life difficult - square brackets do that.

> sd(grades$Grade_Pres[1:35])
[1] 5.55366
> mean(grades$Grade_Pres[1:35])
[1] 66.75429

I couldn't get the two groups compared with a T-test so I did it in Excel.

Found a way to do it.

Pretty easy really:

> t.test(grades$Grade_Dig, grades$Grade_Pres[1:35])

output

Welch Two Sample t-test

data: grades$Grade_Dig and grades$Grade_Pres[1:35]

t = 0.7833, df = 54.334, p-value = 0.4369

alternative hypothesis: true difference in means is not equal to 0

95 percent confidence interval:
-1.301562 2.971073

sample estimates:
mean of x mean of y
67.58904 66.75429

P-value not the same as Excel as I chose to do it for equal variance in Excel.

> t.test(grades$Grade_Dig, grades$Grade_Pres[1:35], var.equal=TRUE)

Two Sample t-test

data: grades$Grade_Dig and grades$Grade_Pres[1:35]

t = 0.8557, df = 106, p-value = 0.3941

alternative hypothesis: true difference in means is not equal to 0

95 percent confidence interval:
-1.099278 2.768789

sample estimates:
mean of x mean of y
67.58904 66.75429

Histograms and box plots

hist(grades$Grade_Pres, breaks=5)

I don't quite understand this breaks term but it does generate some data.

boxplot(grades$Grade_Dig, grades$Grade_Pres)

http://www.r-bloggers.com/summarising-data-using-box-and-whisker-plots/

Visualisation and Data