Rank: 197 based on 310 downloads (last 30 days) and 9 files submitted

Richard Willey

E-mail: Contact Author

Personal Profile:
Professional Interests:

Watch this Author's files

Files
Watch Lists

Files Posted by Richard					View all 1 - 5 of 9
Updated		File	Tags	Downloads (last 30 days)	Comments	Rating
26 Mar 2012		New Regression Capabilities in r2012a Example code to accompany the "New Regression Capabilities" presentation Author: Richard Willey	2012a regression func..., linear regression, regression, logistic regression, nonlinear regression	30	2	5.0 5.0 \| 2 ratings
18 Oct 2011		Computational Statistics: Feature Selection, Regularization, and Shrinkage MATLAB code for "Computational Statistics: Feature Selection, Regularization, and Shrinkage" Author: Richard Willey	sequential feature se...	25	1	5.0 5.0 \| 1 rating
25 May 2011		Data Driven Fitting with MATLAB Demostration code and data sets for the "Data Driven Fitting with MATLAB" webinar. Author: Richard Willey	localized regressionl..., loess, bootstrap, lowess, localized regression, boosting	93	2	3.0 3.0 \| 1 rating
09 May 2011		Exploring Protein-DNA Binding Sites from Paired-End ChIP-Seq Data MATLAB code for the "Exploring Protein-DNA Binding Sites from Paired-End ChIP-Seq Data" webinar. Author: Richard Willey	chipseqbioinformatics...	3	0
25 Apr 2011		Fitting with MATLAB: Statistics, Optimization, and Curve Fitting Demo code and data for the "Fitting with MATLAB" webinar Author: Richard Willey	outlier detection, censored data, deming regression, common slope problem, total least squares, feature selection	39	0	1.0 1.0 \| 1 rating

Comments and Ratings on Richard's Files			View all 1 - 5 of 14
Updated	File	Comment by	Comments	Rating
20 Dec 2012	Fitting with MATLAB: Statistics, Optimization, and Curve Fitting Demo code and data for the "Fitting with MATLAB" webinar Author: Richard Willey	farideh
21 Nov 2012	New Regression Capabilities in r2012a Example code to accompany the "New Regression Capabilities" presentation Author: Richard Willey	Haroch, Eli	Hi, great new function ( for me ), could not find housing.xlsx though.
14 Aug 2012	Data Driven Fitting with MATLAB Demostration code and data sets for the "Data Driven Fitting with MATLAB" webinar. Author: Richard Willey	Weng, Gwen	It requires the data to be dense and not have large gaps.
06 Aug 2012	Data Driven Fitting with MATLAB Demostration code and data sets for the "Data Driven Fitting with MATLAB" webinar. Author: Richard Willey	Boxiang	The author of this file failed to mention that the number of row of the data to be fit must equals to the number of partition 'N' in the function corss validation. The users have to change it themselves.
24 Apr 2012	Introduction to Classification Files and code from Computational Statistics: Getting Started with Classification using MATLAB® Author: Richard Willey	Swati	execellent sir

Top Tags Applied by Richard	1 - 5 of 41
bootstrap, loess, cross validation, localized regression, logistic regression

Files Tagged by Richard					View all 1 - 5 of 9
Updated		File	Tags	Downloads (last 30 days)	Comments	Rating
26 Mar 2012		New Regression Capabilities in r2012a Example code to accompany the "New Regression Capabilities" presentation Author: Richard Willey	2012a regression func..., linear regression, regression, logistic regression, nonlinear regression	30	2	5.0 5.0 \| 2 ratings
18 Oct 2011		Computational Statistics: Feature Selection, Regularization, and Shrinkage MATLAB code for "Computational Statistics: Feature Selection, Regularization, and Shrinkage" Author: Richard Willey	sequential feature se...	25	1	5.0 5.0 \| 1 rating
25 May 2011		Data Driven Fitting with MATLAB Demostration code and data sets for the "Data Driven Fitting with MATLAB" webinar. Author: Richard Willey	localized regressionl..., loess, bootstrap, lowess, localized regression, boosting	93	2	3.0 3.0 \| 1 rating
09 May 2011		Exploring Protein-DNA Binding Sites from Paired-End ChIP-Seq Data MATLAB code for the "Exploring Protein-DNA Binding Sites from Paired-End ChIP-Seq Data" webinar. Author: Richard Willey	chipseqbioinformatics...	3	0
25 Apr 2011		Fitting with MATLAB: Statistics, Optimization, and Curve Fitting Demo code and data for the "Fitting with MATLAB" webinar Author: Richard Willey	outlier detection, censored data, deming regression, common slope problem, total least squares, feature selection	39	0	1.0 1.0 \| 1 rating

Currently Watching
Type	Description
File	SLM - Shape Language Modeling
File	Optimizing breakpoints for Tables
File	Pivot/unPivot
File	MATLAB code for the nonparametric fitting video
File	Introduction to Classification
File	Fitting with MATLAB: Statistics, Optimization, and Curve Fitting

Files Matching Richard's Watch List					View all 1 - 5 of 6
Updated		File	Tags	Downloads (last 30 days)	Comments	Rating
22 Feb 2012		SLM - Shape Language Modeling Least squares spline modeling using shape primitives Author: John D'Errico	hermite, slm, cubic, spline, breaks, knots	137	77	5.0 5.0 \| 42 ratings
25 Apr 2011		Fitting with MATLAB: Statistics, Optimization, and Curve Fitting Demo code and data for the "Fitting with MATLAB" webinar Author: Richard Willey	outlier detection, censored data, deming regression, common slope problem, total least squares, feature selection	39	0	1.0 1.0 \| 1 rating
20 Sep 2010		Introduction to Classification Files and code from Computational Statistics: Getting Started with Classification using MATLAB® Author: Richard Willey	classification, machine learning, naive bayes, treebagger, classificationnaive b..., statistical learning	110	3	5.0 5.0 \| 6 ratings
06 May 2010		MATLAB code for the nonparametric fitting video Use localized regression, cross validation, and the bootstrap for nonparametric fitting. Author: Richard Willey	localized regression, loess, lowess, cross validation, bootstrap	4	0
05 Feb 2010		Pivot/unPivot Group a flat dataset into pivot table and unpivot it back Author: Oleg Komarov	group, unpivot, unstack, stack, reshape, pivot	7	1	4.33333 4.3 \| 3 ratings

Comments and Ratings Matching Richard's Watch List			1-5 of 99
Date	File	Comment by	Comments	Rating
06 Jun 2013	SLM - Shape Language Modeling Least squares spline modeling using shape primitives	John D'Errico	A long time ago, I wrote a tool that would take an existing spline, and rather than simply evaluate the first or last segments of the spline for extrapolation, I added a new segment that had all desired properties, like monotonicity, concavity, endpoint slope or value constraints, etc. Of course the new segments were fully consistent with the old end points of the spline and the shape at that point. If necessary to meet the specified constraints, I added several segments. The interface for this code was similar to that for SLM, with many possible property/value pairs for any possible shape. I'll claim that this tool fully met with the SLM philosophy, in that it encouraged the user to explicitly specify information about the shape of the extrapolated curve. While I'd like to provide such a tool, more important in my opinion is to write a GUI wrapper for SLM. To a large extent you can do that form of extrapolation already, when you first build the curve. Simply specify knots that go out as far as you need the curve to go. The knots need not always be tight up to the end points of the data. (That is the default for SLM, but you can choose your own knots.) This allows you to directly apply any pertinent shape information about that extrapolated region.	Comment only
06 Jun 2013	SLM - Shape Language Modeling Least squares spline modeling using shape primitives	neutrino4242	Dear John, thanks for your rapid answer. Your reference to Mark Twain gave me a new inside view about his mathematical abilities :-). You are right, it's a bit philosophical question and of course, extrapolation may results in unexpected results. But my background is numerical/physical motivated. I'm interest in of deconvolution of a given time series. The frequency response = susceptibility in fourier space is known. It's is well known, extending the data set = padding is mandatory to avoid boundary effects like ringing. In case of image deconvolution, used as deblurring see e.g. R. Liu, "REDUCING BOUNDARY ARTIFACTS IN IMAGE DECONVOLUTION" + google. The suggestion in the book Numerical Recipes, Chapter 13.1.1 is zero padding. This is fine is the data set starts and ends with zeros but fails in all other cases. Zero padding leads to strong ringing at the beginning and end of the deconvoluted time series, independently of the padding length. The reason is the discontinuity in the data set before deconvolution. A better idea is the padding with constants to avoid the discontinuity. Next better idea is a extrapolation in the (unphysical) padded region where is no jump in first and second derivative. If i perform the deconvolution with such extrapolation, the ringing artifacts disappears. Of course, after deconvolution, only the time span without the padding regions in front and the end of the data set has a physical interpretation. I hope it explains my physical/numerical motivation of extrapolation with the first and last spline. BTW: i) I'm a German and my surname is John :-). ii) I use the slmengine mostly to obtain the numerical derivative of noisy data. From my point of view, the slmengine has many advantages in control of the necessary smoothing of the noisy data set, e.g. concaveup or integral. It's not possible to implement such (physical motivated) features in more sophisticated algorithms like higher order methods or Savitzky-Golay-filters.	Comment only
05 Jun 2013	SLM - Shape Language Modeling Least squares spline modeling using shape primitives	John D'Errico	Hi Neutrino, You might call it an inconsistency. I choose to call it a strongly held difference of opinion. Really, it all comes down to my philosophy about extrapolation as opposed to that embodied in PPVAL. I don't let you extrapolate a spline outside of its support using SLMEVAL. Extrapolation does foolish things, just when you least want it to happen. Perhaps my favorite mathematical quote (that hardly anybody else ever seems to know about) is by Mark Twain, from Life on the Mississippi. “In the space of one hundred and seventy six years the Lower Mississippi has shortened itself two hundred and forty-two miles. That is an average of a trifle over a mile and a third per year. Therefore, any calm person, who is not blind or idiotic, can see that in the Old Oölitic Silurian Period, just a million years ago next November, the Lower Mississippi was upwards of one million three hundred thousand miles long, and stuck out over the Gulf of Mexico like a fishing-pole. And by the same token any person can see that seven hundred and forty-two years from now the Lower Mississippi will be only a mile and three-quarters long, and Cairo [Illinois] and New Orleans will have joined their streets together and be plodding comfortably along under a single mayor and a mutual board of aldermen. There is something fascinating about science. One gets such wholesale returns of conjecture out of such a trifling investment of fact.” The point is, extrapolation does nasty things, so SLMEVAL does NOT allow you to do anything but extrapolate as a constant function beyond the support of the spline. PPVAL does do so. That is a problem of PPVAL, IMHO. So what should you do if you truly do NEED to extrapolate? You should have done a better job fitting the spline! Use the capabilities of the SLM tool to fit a spline that goes out as far as you want it to go. Now you have total control over the shape, and you should be monitoring the results to make sure that you get something intelligent, instead of something virtually random as you would get from PPVAL. To put it another way, if you don't know what the curve should be doing out there beyond the data, or will not choose to deal with controlling the extrapolated shape, then you should not be extrapolating your curve out into those nether regions. Again, this is my opinion, but it seems a very logical one, and it is one that is fully consistent with the philosophies of SLM.	Comment only
05 Jun 2013	SLM - Shape Language Modeling Least squares spline modeling using shape primitives	neutrino4242	Dear John, excellent tool, I use it extensively in my daily work with noisy data. A reference in the next paper will be given. May be, i found a tiny inconsistency regarding the extrapolation of data set. Have a look on this sample code: % aim: find a good extrapolation of noisy data, green line in final plot clear all;close all; x=0:0.01:1.5pi; y=sin(x)+rand(size(x))0.1; xinterp=-2:0.01:3*pi; % matlab interp1 is not designed for noisy data, fails yinterp=interp1(x,y,xinterp,'pchip','extrap'); figure(1);plot(xinterp,yinterp); % first try with slm, only the first and last data point are padded in extrapolation yslm=slmengine(x,y,'endconditions','estimate','plot','on'); yslmeval=slmeval(xinterp,yslm); figure(1);hold on;plot(xinterp,yslmeval,'-r');ylim([-3 3]); % second try, this works, calulation of the polynomial in the extrapolated regions ypp=slmengine(x,y,'endconditions','estimate','plot','on','result','pp'); yppeval=ppval(xinterp,ypp); figure(1);hold on;plot(xinterp,yppeval,'-g');ylim([-3 3]);	5
21 May 2013	SLM - Shape Language Modeling Least squares spline modeling using shape primitives	erick	Thank you, John. According to your suggestion, I have fitted my data again, but unfortunately, the results is worse than seperately fitting the two parts of the curve. I made a simple simulation, but I still got the similar result. Would you please take a few minutes to have a look at my simulation data, I will send it to you by email. Thank you. I found the code written by Meyer on this web site "http://www.stat.colostate.edu/~meyer/srrs.htm", but it was written in R code, not matlab. I think I should learn R code these days. Meyer also did not give out how to fit the data with half convex and half concave, it seems still a long way for me to get out. Would you please give me some suggestion? Thank you. I am looking forward to your new reply, Thank you again.	5