This recent paper deals with the “space of infinite possibilities” of ChIP-Seq data analysis, focusing on challenges of differential binding analysis. They have underlined multiple challenges in this area (Introduction, page 2) and compared 14 software packages, using both real-world and simulated datasets.
The observed differences between the results obtained with different methods were not just quantitative but sometimes qualitative, with one subset of methods showing more enriched peaks in condition A (versus B), while another subset – in condition B versus A! This problem, however, was less pronounced (and sometimes eliminated) within a pipeline that used replicates.
In addition, methods showed “huge differences” in the inferred size of the binding sites.
Authors attribute a part of observed inconsistencies between the methods to different data normalization approaches.
Based on their observations, authors present a decision tree for choosing the optimal method, depending on the experimental setup and data properties (Figure 7).
Even if you think that the conclusions of the paper are based on a limited number of datasets and may not be considered final, they provide very handy “cheat sheet” of the considered software tools in the supplementary file. The latter is very clean and helpful, in the case you want to explore the ChIP-Seq analysis options space yourself.