PCA as Metro-Maps & Hierarchical Clustering on Principal Components

The iris dataset is perhaps one of the most famous datasets used to learn and teach statistics and now machine learning. Being curious about this dataset lead me last time to the TableConvert.com  web site that I discussed in my previous post here (and there.) “Metro Maps” Today while revisiting the Wikipedia link for this dataset (Iris_flower_data_set) my… Read More »

TableConvert.com – free tabular data formats converter

This is a repost of my post on the Biochemistry department: tableconvert-com-free-tabular-data-formats-converter Tabular formats When I first came to Madison in 1986 as a postdoc with Biochemistry professor Paul Kaesberg to learn cloning and sequencing, I was surprised when he said that the most important program to work with sequences was the (now defunct GCG) program… Read More »

Bioawk for handling bioinformatics formats

Today I found a new tool: bioawk that was written by Heng Li who also wrote samtools and bwa. I first discovered it ont this blog: bioawk-basics (Bioinformatics Workbooks) There is also a short tutorial on GitHub: github.com/vsbuffalo/bioawk-tutorial I also found a recent docker image, and in fact there are only 2 images on docker hub: lbmc/bioawk updated 2 months ago,… Read More »

A Boy And His Atom: The World’s Smallest Movie

Moving atoms For some reason a paper copy of “Chemical and Engineering News” (November 11, 2019 – Vol 97 Issue 44) ended up in my hands, and I almost missed this fun section named: “30 years of moving atoms: How scanning probe microscopes revolutionized nanoscience” (link.) The article is progressing over time from 1993 til… Read More »

GREP was written overnight – Birth and Name

I use grep very often, and I made-up and acronym that made sense to me: Get REgular ExPression But I discovered this YouTube video that gives an accurate historical recounting of its birth and where its name came  from. See video below, titled “Where GREP Came From – Computerphile” Summary: this comes from the command g/re/p… Read More »

Beyond Docker? Binder 2.0

Binder 2.0 As just an “end-user” of software for data analysis it is sometimes difficult to just catch up with what’s available, and I end-up discovering new things simply browsing the web looking for something specific, and ending up finding something else… that was the case for Docker a couple years ago. Today (Dec 30,2019)… Read More »

STAR index for human genome – overcoming the hardware barriers

Recently I was testing a Docker image to run a container for Next Gen sequencing, a way to test an existing “pipeline” on the first published study of the effect of the Zika virus. (https://hub.docker.com/r/maayanlab/zika/) Running a docker container may provide some ease in reproducibility, but sometimes there are also hardware barrier that need to… Read More »

Down-sampling FASTQ.gz paired ends

Downsampling I have performed a search for creating a set of down-sampled data from an actual  large dataset, and while there are many creative information on BioStar and other forums, I find that the most versatile and easy to use tool would be one recommended on the forums: seqtk which is available on Github: github.com/lh3/seqtk  Quoting… Read More »

Hunting for SRA sequence archives

SRA: Sequence Read Archive The Sequence Read Archive (SRA) makes biological sequence data available to the research community to enhance reproducibility and allow for new discoveries by comparing data sets. The SRA stores raw sequencing data and alignment information from high-throughput sequencing platforms, […] However, it is rather difficult to even find the download links… and even… Read More »