How to remove February 29th for leap years in a daily time series over 43 years?

I have an array of values (x) representing daily data for 43 years (1979-2021) where size(x)=15706

I rearranged the data to have a matrix with years as rows and days as columns.

% some x --> keep only multiple of 365 days (removes last 11 days)
x           =   x(1:365*43).';

% reshape x to have years as rows and days as columns
x_matrix    =   reshape(x,365,[]).' 

However, this is problematic for leap years since now my columns in x_matrix are not aligned with a given day. Indeed, x_matrix(year n, 1) is not always January 1 but is shifted toward later in January as years pass.

My data represents year 1979-2021 where 1980, 1984, 1988, 1992, 1996, 2000, 2004, 2008, 2012, 2016, 2020 are leap years.

How can I remove the last day of February for each leap year to have x_matrix align for each year with a given day where x_matrix(year n, 1) is always January 1?

Is there a useful function for that somewhere?

Thank you

[dpb]

You could, but why create an artificial structure that doesn't represent reality and throws away perfectly good data?

Create/use a timetable instead and you can then process/access the data however you choose with grouping variables one of which can easily be the month, day, etc., and then having to access a day by indexing into a column becomes a thing of the past. There's also a way to reference the day of the week instead of numeric day of month if that's another way want to look at the data.

IOW, keep all the info, just slice 'n dice as needed from the data values themselves, not by an artificial rearrangement.

You may want to look into "splitapply workflow" and grouping variables and in particular with the timetable class the retime and rowfun functions. They're amazingly powerful at such data to answer all kinds of questions in very compact code. Virtually all the addressing issues just melt away...

[dpb]

As for the Q? actually asked, although it's not the approach I would recommend, my utility function isleapyr may be of value (although I hope you'll not use it as you've outlined above).

function is=isleapyr(yr)
% returns T for input year being a leapyear
% dpozarth -- from dark ages; modified to handle new datetime class

  if isdatetime(yr), yr=year(yr); end
  is=(eomday(yr,2)==29);
end

[A LL]

Thank you dpb for your answer.

I want to do a pcolor of my x_matrix to see a tendency in the seasonality and years of my data so I need to have 365 days for years for non-leap and leap years to do that (see attached figure).

Therefore, I am looking for a way to remove that extra day on Feb 29 for leap years only so that x_matrix(year n, 1) is always January 1.

[dpb]

OK, that's one reason I hadn't thought of for a rectangular array...if you don't want to regularize with missing data for the non-leap years, then simply

t=datetime(1979,1,1):days(1):datetime(2021,12,31);
isNotLeapDay=~(month(t)==2 & day(t)==29);
data=data(isNotLeapDay);

That's assuming a vector of the data before trying to reshape into an array.

I think I'd still keep the full data set and just extract as above for the special purpose...

[A LL]

Thank you!

Answer 1

[Jon]

I fully support @dpb suggestions regarding working with timetables. However for purposes of making your graphic, you could do this

% define example data
x = rand(15706,1);

% make datetimes corresponding to data
t1 = datetime(1979,1,1);
t2 = datetime(2021,12,31);
t = t1:t2;

% find all the leap days and remove them
idl = month(t)==2 & day(t)==29;
xClean = x(~idl);

% make matrix with rows as days and years as columns
X = reshape(xClean,365,43)

[Steven Lord]

Another way to do this, assuming you just know how many rows the data has and what the initial date is (without knowing the ending date ahead of time) you could use a datetime and a duration or calendarDuration array.

t1 = datetime(1979,1,1);
t2 = datetime(2021,12,31);
tv1 = t1:t2; % start and end known
n = 15706;
tv2 = t1 + days(0:n-1); % start and number of steps known
tv3 = t1 + caldays(0:n-1); % start and number of steps known
tv1(end), tv2(end), tv3(end)

ans = datetime
   31-Dec-2021

ans = datetime
   31-Dec-2021

ans = datetime
   31-Dec-2021

isequal(tv1, tv2)

ans = logical
   1

isequal(tv1, tv3)

ans = logical
   1

Now locate leap days as before.

isleap = @(dt) month(dt) == 2 & day(dt) == 29;
tv1(isleap(tv1)).'

ans = 11×1 datetime array
   29-Feb-1980
   29-Feb-1984
   29-Feb-1988
   29-Feb-1992
   29-Feb-1996
   29-Feb-2000
   29-Feb-2004
   29-Feb-2008
   29-Feb-2012
   29-Feb-2016
   29-Feb-2020

Answer 2

[James Tursa]

To get rid of the leap days, you can use evenly spaced indexing since the number of days between leap days is constant for your time span. Since index 1 corresponds to Jan 1, 1979, that means the first leap day is Feb 29, 1980 which is index 365+31+29 = 425. The next leap day will be 4*365+1 = 1461 days later. So the code to get rid of the leap days for your data is (do this prior to reshaping):

x(425:1461:end) = [];