r - How to apply big data on this p-value corrgram? -


i studying didzis' p-value corrgram different input data examples, insignificant p-value (p < 0.05) corresponds perfect curve fit, strange, see fig 1-3.

fig. 1 output of "extreme" input data #1, fig. 2 output minimum input data #2, fig. 3 output didzis' input data #3,

enter image description here enter image description here enter image description here

statistical inspection.

  • fig. 1 p-values high when r small,
  • fig. 2 p-values high confidence intervals wide, not sure if drawing graph there appropriate,
  • fig. 3 low p-values when curve fitting perfect - observation can confusing

input data test cases

real live data example #1 "extreme" example , application output in fig. 1 

## 1 make list of lists set.seed(24) a=541650 m1 <- matrix(1:a, ncol=4, nrow=a) str(m1)  a=360; b=1505; c=4; m2 <- array(`length<-`(m1, a*b*c), dim = c(a,b,c))  res <- lapply(seq(dim(m2)[3]), function(i) cor(m2[,,i])) str(res)  res <- lapply(res, function(x) eigen(replace(x, is.na(x), 0))$vectors[,1:1])     str(res)

minimum example #2 , application output in fig. 2 

a <- 1505 res <- list(rnorm(a), rnorm(rnorm(a)), rnorm(rnorm(rnorm(a))), rnorm(rnorm(rnorm(rnorm(a))))) str(res)

standard input example didzis used election data #3 in fig. 3

res <- usjudgeratings[,c(2:3,6,1,7)]

to make p-value corrgram

## 2 didzis https://stackoverflow.com/a/15271627/54964 panel.cor <- function(x, y, digits=2, cex.cor) {   usr <- par("usr"); on.exit(par(usr))   par(usr = c(0, 1, 0, 1))   r <- abs(cor(x, y))   txt <- format(c(r, 0.123456789), digits=digits)[1]   test <- cor.test(x,y)   signif <- ifelse(round(test$p.value,3)<0.001,"p<0.001",paste("p=",round(test$p.value,3)))   text(0.5, 0.25, paste("r=",txt))   text(.5, .75, signif) }  panel.smooth<-function (x, y, col = "blue", bg = na, pch = 18,                         cex = 0.8, col.smooth = "red", span = 2/3, iter = 3, ...) {   points(x, y, pch = pch, col = col, bg = bg, cex = cex)   ok <- is.finite(x) & is.finite(y)   if (any(ok))     lines(stats::lowess(x[ok], y[ok], f = span, iter = iter),           col = col.smooth, ...) }  panel.hist <- function(x, ...) {   usr <- par("usr"); on.exit(par(usr))   par(usr = c(usr[1:2], 0, 1.5) )   h <- hist(x, plot = false)   breaks <- h$breaks; nb <- length(breaks)   y <- h$counts; y <- y/max(y)   rect(breaks[-nb], 0, breaks[-1], y, col="cyan", ...) }  data <- res str(data)  pairs(data,           lower.panel=panel.smooth, upper.panel=panel.cor,diag.panel=panel.hist)

about significant upperbound

the source says study not statistically siginificant 15k points may become significant 2-3m points. observation becomes signifant 6-7m data sample , study, data 541650 541650 6925867. think there no problem in plotting big data sets in didzis' p-value corrgram in theory. algorithms making possibly simplifications, or causing clusterisation of points such many figures increasing diagonal or y=0 line.

os: debian 8.5
r: 3.3.1


-

Comments

Post a Comment

Popular posts from this blog

java - SSE Emitter : Manage timeouts and complete() -

i writing web app multiple listeners(evcentsource sse clients js) connected server . want store sse emitter per connected listener : can done in memory or other means assigning id each client able achieve far now question ; how send response/event specific client connected web app ? while doing sseemmiters stored either getting completed or timed out. how prevent ? how keep sseemmiter available open infinite time (till client closes) , send events selectively .
Read more

jquery - uncaught exception: DataTables Editor - remote hosting of code not allowed -

i'm trying implement crud operation using datatableseditor using datatables.but i'm getting error messages: 1)uncaught exception: datatables editor - remote hosting of code not allowed. please see http://editor.datatables.net details on how purchase editor license 2)typeerror: a.editor undefined 3) $.fn.datatable.editor not constructor" what reason? here configuration: <%@ page language="java" contenttype="text/html; charset=iso-8859-1" pageencoding="iso-8859-1"%> <!doctype html public "-//w3c//dtd html 4.01 transitional//en" "http://www.w3.org/tr/html4/loose.dtd"> <html> <head> <meta http-equiv="content-type" content="text/html; charset=iso-8859-1"> <title>insert title here</title> <link rel="stylesheet" type="text/css" href="//cdn.datatables.net/1.10.12/css/jquery.datatables.min.css"> <link re...
Read more

java - How to resolve error - package com.squareup.okhttp3 doesn't exist? -

i trying convert json java object use of gson in maven, following youtube video guidance - https://www.youtube.com/watch?v=vqgghm9pwe0 , theres error occurs when in main class error - package com.squareup.okhttp3 doesn't exist. code below: java package com.codebeasty.json; import com.squareup.okhttp3.okhttpclient; public class main { private static okhttpclient client = new okhttpclient(); public static void main (string [] args) { } } i put in dependency in pom.xml: <dependencies> <dependency> <groupid>com.squareup.okhttp3</groupid> <artifactid>okhttp</artifactid> <version>3.4.2</version> </dependency> <dependency> <groupid>com.google.code.gson</groupid> <artifactid>gson</artifactid> <version>2.8.0</version> </dependency> </dependencies> i dont understand why doesn't recognize com.squareup. there may need download? have downl...
Read more