What is missing from MATLAB #2 - the next decade edition
Rik
on 31 Jul 2020
Latest activity Reply by Walter Roberson
on 18 Oct 2023
Meta threads have a tendency to grow large. This has happened several times before (the wishlist threads #1 #2 #3 #4 #5, and 'What frustrates you about MATLAB?' #1 and #2).
No wonder that a thread from early 2011 has also kept growing. After just under a decade there are (at time of writing) 119 answers, making the page slow to load and navigate (especially on mobile). So after a friendly nudge; here is a new thread for the things that are missing from Matlab.
Same question: are there things you think should be possible in Matlab, but aren't? What things are possible with software packages similar to Matlab that Matlab would benefit from? (note that you can also submit an enhancement request through support, although I suspect they will be monitoring activity on this thread as well)
What should you post where?
Next Gen threads (#1): features that would break compatibility with previous versions, but would be nice to have
@anyone posting a new thread when the last one gets too large (about 50 answers seems a reasonable limit per thread), please update this list in all last threads. (if you don't have editing privileges, just post a comment asking someone to do the edit)
168 Comments
I'm sure I mentiond this at some point... but it would be useful if you could substitute a logical index for more than one dimension.
For example:
img = imread('flamingos.jpg');
intens = rgb2gray(img);
mask = intens > 128;
newimg = img;
At this point, we would like to be able to use something like
%newimg(mask, 3) = 255 - img(mask, 3);
but that is not going to work, and instead we need to proceed something like
temp = img(:,:,3);
temp(mask) = 255 - temp(mask);
newimg(:,:,3) = temp;
subplot(2,1,1); image(img); title('original');
subplot(2,1,2); image(newimg); title('modified')
Now, there is a way to do it without a temporary variable, but it is ugly...
img = imread('flamingos.jpg');
intens = rgb2gray(img);
mask = intens > 128;
newimg2 = img;
newimg2(find(mask) + 2 * numel(mask)) = 255 - img(find(mask) + 2 * numel(mask));
figure;
subplot(2,1,1); image(newimg2); title('modified -- indexing')
isequal(newimg, newimg2)
An AI copilot feature-option in the IDE, where either matlab has their own model (optimal) with adjustable privacy that users can determine (to avoid possible gdpr etc issues), and/or the capacity to configure the copilot to work with an LLM service of our choice (also possibly run locally).
In the current state of afairs, continuing using an ide with no copilot integration is non-sustainable. Not for the future, for like yesterday. I already feel quite anxious falling behind compared to python users in terms of copilot language-specific support. There is a ton of python stuff out there that LLMs are trained on, and ime existing LLMs are worse in many other languages, including matlab. The direction copilots seem to get is different "specialised experts", and it seems finetuning different models to specific languages each has potential.
It would be great if Mathworks finetuned an LLM, which could, for example, either be some version of GPT3.5/4 that we could then use through an API, or a version of codellama that is (almost) open source that then mathworks itself could provide through an API (as an extra toolbox, or subscription, or whatnot). Or both options (why bet on one horse only?) or something else entirely. Also, it would be great if people who write matlab using other environments like neovim or vs-code had access to that model too. Even better if it is available for people to also run locally, like the codellama derivatives.
Finetuning a model should not be a great deal for a big company like mathworks, people with much less budget do this sort of thing nowadays. Mathworks has access to tons of matlab code they could use, and I cannot see anything putting any great obstacle to that apart from intention. There are a lot of companies and startups right now doing this sort of thing for other languages. But if mathworks does not do that with matlab, I don't know if anybody else will bother with it. So please consider offering something in that direction, because very soon it seems that writing code without option for copilot assistance will not be conceivable.
In some sense, you already deserved so even before.
I wonder if I will get any residuals for the various models that train on the numerous answers I have given? Somehow I suspect not...
When defining Abstract methods on a class, I would like to be able to fix the arguments on the abstract class.
Something like this would be usefull:
classdef AbstractSuperClass
methods (Abstract)
function AbstractMethod(self,input1,input2)
arguments
self
input1 (1,1) double
input2 (1,1) double
end
% no function body because it is abstract
end
end
end
Right now, I always end writing two versions of the same method to avoid the need to copy the arguments validation block to the subclass:
classdef AbstractSuperClass
methods
function Method(self,input1,input2)
arguments
self
input1 (1,1) double
input2 (1,1) double
end
abstract_version_of_method(self,input1,input2)
end
end
methods (Abstract,access=protected)
abstract_version_of_method(self,input1,input2)
end
end
but this feels wrong to do it and results in programming errors because I forget what the input signature of the method is, the above syntax would be more elegant and would allow for checking whether the input signature of the implementation of the method follows the definition in the abstract class.
With the new arguments (output), I would even be able to fix the output type of my abstract function
Oooh, tentative +1 on this. This is something I hadn't even thought of, and I'm not sure what all the implications of doing this in Matlab would be. But it's basically how traditional static OOP languages like Java and C++ work, and there it's obviously the right thing and we just take it for granted. There, the types etc of the input arguments are part of the interface or signature of those methods, and subclasses must conform to them, and that just makes things easy to reason about. (IMHO.)
I wish clearvars had an optional "do-nothing" flag a la the /L switch in CMD XCOPY to display the variable that would be cleared with the given variables list...would let one confirm a wildcard expression didn't accidentally wipe out something wished to have kept; particular with the -except clause.
Oooh, good idea. PowerShell does this for a bunch of their cmdlets like xcopy. I call this a "dry-run" or "what-if" run. I think PowerShell and recent Windows have standardized on the -WhatIf option for this.
Maybe there's a way I've not found, but I wish arrayfun and cellfun would have automagic argument expansion so one could pass other arguments to the anonymous function without having to replicate them manually to match the others. This would add greatly to the convenience in using either; the present working example happens to be building a set of target range expressions to stuff into a cell array that will be written to Excel although that really has nothing to do with the request/enhancement, just happens to be current time was frustrated that there's no way to pass constants to the anonymous functions to allow them to be generalized.
Example:
Building a variably-sized workbook where it is desirable that the sums over a section of the sheet be formulas rather than the current fixed constant value of the data as the sheet will subsequently be modified by hand; the tool is to build the original working pattern by compending various data sources and arranging for the end user...
It boils down to the point at which one has a set of row range indices and a set of columns over which to build the formula and insert into the cell array which is subsequently written to the workbook. That code looks something like
% anonymous function that builds Excel =SUM(r1:r2) expression for given row range, column
xlsSumRange=@(r1,r2,c)strcat('=SUM(',xlsAddr(r1,c),':',xlsAddr(r2,c),')');
% typical use
col=xlsCol2Col('G'); % another internal translation layer of local array position from Excel column
cOut(isTotal,col)=arrayfun(@(r1,r2,c)xlsSumRange(r1,r2,c),RS1,RS2,repmat('G',numel(RS)-1,1),'UniformOutput',0);
col=xlsCol2Col('K'); % another internal translation layer of local array position from Excel column
cOut(isTotal,col)=arrayfun(@(r1,r2,c)xlsSumRange(r1,r2,c),RS1,RS2,repmat('K',numel(RS)-1,1),'UniformOutput',0);
This works, but the expression "repmat('G',numel(RS)-1,1)" needed is really inconvenient and clutters up the code legibility greatly. I've had any number of similar case in the past where an anonymous function is useful shorthand but then to use more than once requires a workaround like the above.
The alternative is to redefine the anonymous function dynamically and embed the constant inside it for the given invocation.
To visualize, an example output for the above for one invocation looks like for the call with column 'G'
K>> arrayfun(@(r1,r2,c)xlsSumRange(r1,r2,c),RS1,RS2,repmat('K',numel(RS)-1,1),'UniformOutput',0);
ans =
3×1 cell array
{'=SUM($G$3:$G$17)' }
{'=SUM($G$22:$G$372)' }
{'=SUM($G$377:$G$414)'}
K>>
The above could be in a loop over the number of columns instead of using the explicit columns, but was part of still rearranging the output file structure at the time...and, haven't taken the time to clean it all up yet...
+1 on this, @dpb. This "implicit scalar expansion for (array|cell)fun args" is a use case I've had occasional cause to want, and haven't been able to figure out a good way to do, except for ditching arrayfun and Functional Programming style and inverting the logic/control-flow to just write a vectorized or "regular Matlab" version of the function I'm trying to do.
In other use cases, as long as I don't care about method-call overhead, I've been able to wrap the value to be expanded in an "infinite copies" array, like this SPAM thing here:
classdef SPAM
% "spam" a value to every element of an infinite virtual array
properties
Value
end
methods
function this = SPAM(x)
this.Value = x;
end
function out = subsref(this, S) %#ok<INUSD>
out = this.Value;
end
end
end
But that doesn't seem to work with cellfun/arrayfun, because it looks like they do an explicit "size(...)" test on their inputs before trying to index in to them.
>> x = magic(3);
>> z = arrayfun(@plus, x, SPAM(420))
Error using arrayfun
All of the input arguments must be of the same size and shape.
Previous inputs had size 3 in dimension 1. Input #3 has size 1
>> spam = SPAM(420);
>> z = NaN(size(x)); for i = 1:numel(z); z(i) = x(i) + spam(i); end
>> z
z =
428 421 426
423 425 427
424 429 422
>>
Can't say I blame Matlab for this, though. I like explicit size-conformity checks; catches bugs more often than it prevents features.
Thanks @Stephen23, @Bruno Luong and @Steven Lord for the feedback and example coding...I'll snag the code snippets and stuff away with the present code in a comment block and come back and revisit when some of the time pressure lessens...
You could avoid the need for 'UniformOutput' by using string arrays. I also changed xlsSumRangeCol to use string concatenation (with +) rather than strcat.
xlsAddr = @(r,c) sprintf("$%c$%d", c, r);
xlsSumRangeCol = @(c) @(r1,r2) "=SUM(" + xlsAddr(r1,c) + ":" + xlsAddr(r2,c) + ")";
c = 'G';
xlsSumRange = xlsSumRangeCol(c);
RS1 = 1:3;
RS2 = RS1+10;
arrayfun(xlsSumRange,RS1,RS2)
Or you could vectorize xlsSumRangeCol to eliminate the need for arrayfun.
xlsSumRangeCol = @(c) @(r1,r2) "=SUM($" + c + "$" + r1 + ":$" + c + "$" + r2 + ")";
xlsSumRange = xlsSumRangeCol(c);
xlsSumRange(RS1, RS2)
Or if you want the range to always be a certain number of elements long:
xlsSumRange1 = @(c, r1, n) "=SUM($" + c + "$" + r1 + ":$" + c + "$" + (r1+n) + ")";
xlsSumRange1(c, RS1, 10)
@dpb The whole point of TMW anymous function design is to be able to call it with only changing variables, and constant variable in the workspace is captured in the context without explicitly entered as input argument. This logic is tailored with arrayfun/cellfun, and other functions that requires function handles as input (fmincon and friends).
Or if you want it properly you could cascade the anonymous functions as following
xlsAddr = @(r,c) sprintf('$%c$%d', c, r);
xlsSumRangeCol = @(c) @(r1,r2)strcat('=SUM(',xlsAddr(r1,c),':',xlsAddr(r2,c),')');
c = 'G';
xlsSumRange = xlsSumRangeCol(c);
RS1 = 1:3;
RS2 = RS1+10;
arrayfun(@(r1,r2)xlsSumRange(r1,r2),RS1,RS2,'UniformOutput',0)
Admitly the 'Uniform' argument does look ugly and for-loop is much more readable.
Possibly, Walter.
In the end I gave up the arrayfun syntax entirely in this application because there are M rows and N ranges per row over which to build a compound Excel range so the number of elements in the argument list isn't the same and the complexity just got to be too much.
So, I ended up reverting to a doubly-nested loop using the direct sprintf() form for each combination and then join'ed them in the end to pass to Excel COM instruction...sometimes we fall into the trap of trying to remove explicit loops too often.
But, I STILL think the idea has merit...there have been a number of times I'd really, really like to have been able to have done.
I wonder if string operations would be faster these days?
xlsSumRange = @(r1,r2)"=SUM(" + xlsAddr(r1,c) + ":" + xlsAddr(r2,c) + ")");
in which case you would not need 'UniformOutput', 0 directly because string arrays are a thing. (On the other hand you cannot store a string array into a scalar location.)
Somehow I developed an aversion to redefining the function to parameterize; I guess on due consideration it would be the way to simplify with current syntax. Thanks for the prod, @Stephen23...but, I still think the suggestion would be a worthwhile extension to the [array/cell]fun syntax.
This just boils down to function parameterization, which is indeed simpler and more efficient:
c = 'G';
xlsSumRange = @(r1,r2)strcat('=SUM(',xlsAddr(r1,c),':',xlsAddr(r2,c),')');
% typical use
col = xlsCol2Col(c);
cOut(isTotal,col) = arrayfun(xlsSumRange,RS1,RS2, 'UniformOutput',0);
Replacing STRCAT with SPRINTF would also be more efficient:
xlsSumRange = @(r1,r2) sprintf('=SUM(%s:%s)',xlsAddr(r1,c),xlsAddr(r2,c));
read/writetable from/to Excel workbooks should be able to return/write the comment (now called something else, I forget what) field and the formula associated with the cell as well as the value. While one can write COM to do so, it then takes either using COM entirely (can be a lot of work) or have to use both high-level and COM on the same file; either of which options isn't ideal. Sure, one could forego Excel entirely since have MATLAB, but unfortunately can't always have what we want in that regards.
One can mung on the old xlsread/write routine and add stuff into it, but it's also kinda' klunky and while more convenient in some ways as compared to writing all COM totally from scratch isn't nearly as convenient as would be a higher-level interface with the other current toolset.
There's a hook to a function call in xlsread although I was never able to get it to work to do either of the above; because the Excel COM object isn't exposed to the function to be able to make direct access from that function.
"matlab coder" toolbox can't produce only one C/C++ file, refer to:
matlab AXES PROPERTIES missing:
I'm going to be a weirdo and say that I would like it if imshow() supported IA and RGBA inputs. I'm probably the only person who ever uses RGBA workflows in MATLAB, but these are wishes we're talking about. I might as well wish for a function that gives me a sandwich.
I already made my own tool for MIMT, but it kind of drives me up the wall to not be able to use all the conveniences I created when answering questions on the forum. I feel like I'm defeating myself.
Probably it breaks compatibility, but it would make functions/operators svd, *, ', .' dealing with n-d arrays similar to pagesvd, pagemtimes, ... followed by appropriated reshape.
Extend the pagexxx with matrix left/right division, lu, qr, chol, eig, etc...
@Adam Not only the floating point rounding can create different numerically result, but what matters even more is that the floating point counts when multiply matrices can be different.
Consider the product
A*B*x
with
- A: p x m,
- B: m x n,
- x: m x 1
with p = 100, n = 100, m = 1000,
It is much cheaper to perform
A*(B*x) than (A*B)*x.
And in general the less operations is carried out, the less round-off has effect on the final result.
Ah, that's the thing I was missing, thanks for clarifying that
@Adam: binary floating point addition and multiplication are not associative:
This means the order matters.
I don't really understand why associativity control is important.
Is the result not the same for both cases?
Maybe I'm missing something.
Good idea
pagemtimes(A,B,C)
User should be able somehow to control the associativity, meaning equivalent to
pagemtimes(A,pagemtimes(B,C))
or
pagemtimes(pagemtimes(A,B),C)
I really find these page functions very usefull. Some usefull additions in my opinion would be:
Have the option to multiply more than two matrices with pagemtimes
I now always end up writing pagemtimes(A,pagemtimes(B,C)). It would be very usefull to be able to write pagemtimes(A,B,C) in the future for the readability of the code.
Paged version of decomposition
The decomposition is also a usefull tool and would be nice to have a page version of it available
Thank you Steve, and also to the dedicated developpers.
We only added one page function in release R2022b. It's not on your list, but you might find pagenorm useful.
Hi Steve, the priority remains in this order to me. Thanks
The pagemldivide, pagemrdivide, and pageinv functions are new as of release R2022a. We also have several other page* functions (including all but one of the items in your top 4 levels, we don't have a pageeig function) as shown in the functions list.
So does your list still start with pageeig or have one of the items at a lower level jumped ahead of it in your prioritization?
Priority in term of usefullness IMO should be
- pagetranspose, pagectranspose
- pagemtimes
- pagemrdivide (pagemldivide can just be a wrapparound), but as mrdivide is based on qr factorization, interm of dev I guess TMW woul need to make dave of pageqr first.
- pagesvd, pageeig
- pageqr, pagelu, pagechol
- pageldl
- pageexpm
- pagehess, pageqz, pageschur
- pageinv, pagetrace, pagedet
You've listed a bunch of functions there. Out of curiosity, if you had to prioritize which additional functions would you like to see get the pagemtimes, pagetranspose, pagectranspose, and pagesvd treatment first? What would be your top say half a dozen?
Being able to access the properties/methods/fields of an object/struct even after indexing into one would be a big deal.
Currently, if you have a struct A, with fields size and count, you could do A. then tab to show the options for the fields. But if you do A(1). you cannot now tab to show the options for the fields. Same with an object. So the discoverability is gone.
I want to be able to index into an object/struct and still have the available fields/properties/methods for that parent show up.
I just tried this on 2020b and hitting tab after A(1). showed the fields.
+1 on this: Currently, if you have a struct A, with fields size and count, you could do A. then tab to show the options for the fields. But if you do A(1). you cannot now tab to show the options for the fields. Same with an object. So the discoverability is gone.
Maintain dimensions when getting a field of a class.
If Data is a 3x2x4 array of objects/structs with a field "foo", then Data.foo should be an 3x2x4 array of the type of "foo". Not a 1x24 array, or is it the other way around.
Yes you are right, now I remember. Always have to do something like "foo = [Data.foo]".
It currently is neither: it's a comma separated list.
for i1=1:3,for i2=1:2,for i3=1:4,s(i1,i2,i3).Data=rand;end,end,end
s.Data
To do something with it you'll have to capture it with [] or {} or a function that allows arbitrary input.
I agree. Makes intuitive sense to me and would be convenient for extracting foo.
Fix the semantics of "clear".
"clear all" does not clear all, but "clear classes" does. Go figure. I want a way to clear classes and only classes.
Clear also cannot clear specific classes in different packages if they have the same name. I.e. pkg1.my_class, and pkg2.my_class. Matlab's package system is broken, but at least some things could be improved.
In the code analyzer, there is no easy way to find what #ok directive controls a particular flagged issue.
(And no overall list of the available #ok directives, at least that I can find. Even the enable/disable messages in preferences don't show them.)
Perhaps the codeIssues function introduced in release R2022b will be of use to you. The CheckID variable in the issues table stored in the codeIssues object is the identifier you'd use in the %#ok pragma to suppress Code Analyzer's reporting of that particular issue.
Maybe I've only looked in the wrong places, but the changes from release to release about what is marked as a warning by mlint are poorly (or not) documented in the release notes.
So there are more things leaving room for improvement regarding this.
I would like imagesc(C) to work with 3 dimensional arrays.
vol 3d v2 in File Exchange.
See also the code I posted in https://www.mathworks.com/matlabcentral/answers/1447449-how-to-draw-a-volxe-size#comment_1728709 about a month ago, which creates voxel cubes (taking care that each one has proper face orientation so normals can be calculated).
The disadvantage of the code I posted is that it builds a cube for each voxel, instead of trying to merge together adjacent voxels to reduce the drawing cost. Drawing one cube for each voxel has advantages if you want to be able to use per-voxel alphadata (since two adjacent cubes with exactly the same "value" attribute might potentially be assigned different alpha.
The code I posted there only draws cubes for "occupied" voxels. Which reduces drawing costs, but leads to questions about the best way to create alpha data -- do you create the alpha data as a cuboid that occupies the full possible space, or do you create the alpha data in a way that maps only to the occupied locations?
Sorry about the name typo.
Since it is a very different goal, why should it have the same name? I'm a firm advocate for the concept that a function should do one Thing. What constitutes a Thing can of course be flexible, but in this case 2D and 3D are different enough that they would require different tools.
Have you looked at the File Exchange for 3D viewers? There are many of them. One might already do what you want. I don't recall any built-in function that will directly do this. The colors are simply based on the current colormap, so that should be easy enough to implement.
Also, next time you use your phone to answer here, double check the spelling of the name of the person you're replying to.
Hi Rik. I would use it as a volumetric viewer as you say. I would pass a 3D array to imagesc (or you can call it something else) and it would be able to plot 3D little cuboids, instead of 2D faces it plots now. The cubes would be colored based on the values stored in the array, in the same way imagesc works now. Maybe I would also use AlphaData to make some of the little cuboids transparent based on what I am interested in.
What should the behavior be? Why aren't you using a dedicated volumetric viewer for volumetric data? And if you're talking about RGB-images, what would be the point of using imagesc instead of image (or the high-level imshow)?
Your TypicalX option for fminunc() (and others?) should be extended beyond usage for gradient evaluation only.
I have used normalized variables for optimization for a long time. By default, I do start with [1,1,...,1] as you do, but for gradient evaluation only. There are many areas where a spread of variables is vast (radiolectronics: kiloohms to nanofarads).
Currently evalin() accepts a context argument (such as 'caller' or 'base' or symengine()), and a character vector that is the command to be executed.
This runs into the same horrors as using eval() .
It is difficult to convince people to give up using eval(); they tend to think that their particular use case makes it necessary (it rarely is!). And the cause is Not Helped At All when we have to say "well, it is true that the closely-related "evalin() is needed sometimes"
It would therefore help if there was a way to do something like evalin() but with a function handle. For example,
evalin('caller', 'whos')
might become something like
evalfcnin('caller', @whos)
To be honest, I do not know how this would work in practice, considering that functions need their own workspace to execute in. Maybe some of the technology behind shared variables and nested functions could be used, so that the function could have read/write access to the designated function workspace and yet still be able to have its own private variables.
Sometimes I want to be able to get a look at persistent variables in a function that is not the caller; I have never found a hint that is possible. But the existence of persistent variables that go away when you "clear" the function, implies that each parsed function already has some kind of workspace associated with it. And sometimes it seems to me that it would be useful if you were able to get a referene to that workspace and tell a function to execute using that workspace. Something that might look like
evalinWorkSpace( matlab.workspace.getinstance.caller, @whos)
Having workspaces as accessible objects leads to some interesting possibilities about saving and restoring state, such as for the purpose of recovering from power failures; it also heads towards possibilities such as co-routines.
@Paul In your last example, a shared-data-copy of d is passed into func( ). Inside the function, when b gets modified, is when a deep copy gets made.
Pass-by-reference would allow direct modification of the data, but it's my understanding that, in general, Matlab doesn't allow direct modification of the data (with one exception that I know of). So in a function such as this
c = func(d)
function a = func(b)
a = 2*b
end
the computation of a operates on the same data that is addressed by d in the caller's workspace. So even though func doesn't directly modify d, we still get the advantage of not having to create and delete a copy.
But in this situation
c = func(d)
function a = func(b)
b = 2*b;
a = b
end
the input to func is passed-by-value, i.e., a copy of d is passed into func and then deleted when func completes executing.
Is my understanding of the basic mechanisms for passing data in to functions incorrect?
pass-by-reference, "where the parameter in the callee becomes an alias to the variable in the caller's workspace" allows direct modification of the data .
Consider for example the C call setup. You can pass numeric scalars by value. You can pass struct (themselves) by value. But when you pass a vector or array, what gets passed is the address, not a copy of the content. If the called function does not declare "const" then it is permitted to use the pointer to modify the original data in-place.
I thought Matlab uses pass-by-reference unless the parser determines that the receiving function modifeds the input argument?
For doing this sort of thing, I like to lean on the correspondence between workspaces and structs: both are a collection of name/value pairings, and support the same set of valid identifiers for variable names or field names. Package a workspace state up into a struct, pass it to a function that takes a struct input, unpackage that struct into the callee function's workspace, and work with them there, pass the results as a struct, and then optionally unpackage them in the caller's workspace. I use a pair of vars2struct and struct2vars helper functions to do this.
function foo()
x = 42;
y = "something";
% One-way exchange:
bar(vars2struct);
% Round-trip exchange, modifying original caller's variables:
struct2vars(baz(vars2struct));
end
function out = bar(s)
struct2vars(s);
out = sprintf('x=%d, y=%s', x, y);
end
function out = baz(s)
struct2vars(s);
x = x + 3;
y = y + " more";
out = vars2struct;
end
This might be a little cleaner than granting a function direct access to another function call's workspace. Cross-stack-frame operations can get messy. And it gives you nice "transactional" behavior, where either all or none of the work in the child function gets done.
But I don't think it supports coroutines.
Adding true pass-by-reference function arguments (where the parameter in the callee becomes an alias to the variable in the caller's workspace) to Matlab could be another way of handling this. Though I don't know if that's a good idea: then you get shared mutable state in ways that aren't always obvious from looking at the function call site, and takes you further away from Matlab's pass-by-value, sort-of-functional-programming paradigm.
Here's the helper functions:
- vars2struct: https://github.com/janklab/janklab-core/blob/master/Mcode/toplevel/vars2struct.m
- struct2vars: https://github.com/janklab/janklab-core/blob/master/Mcode/toplevel/struct2vars.m
I think it might be cool and useful for debugging, but tempting and dangerous, to have dbstack include the workspaces of each stack frame in its output.
I'd like questdlg() to be able to take more than 3 buttons.
Often I need 4 buttons, like "Yes", "No", "Cancel", and "All", like if I'm looping over files in a listbox asking if you want to delete them or not. Why can't questdlg() take any number of arguments, much like other functions like ones(), and just put up the buttons in a dialog box in an array? The last argument could be the button that you'd like for the default (as it is now).
I know I can still use menu() but it's deprecated in favor of listdlg() and I'd rather have buttons than a listbox - it's just more intuitive.
If you're using uifigure objects you can create a uiconfirm object for your uifigure. This code won't run in MATLAB Answers but it will in MATLAB Online or desktop MATLAB.
f = uifigure;
y = uiconfirm(f, 'Select an option', 'Option Selector', ...
Options=["Yes", "No", "Cancel", "All"])
Adaptive specification of feature map output size:
Single and multi-objective vision tracking algorithms:
KCF, GOTURN, FairMOT, deepSort ,... are some of the best and successful algorithms in recent years, but unfortunately, why is there no such implementation in the "sensor fusion and tracking toolbox", "deeplearning toolbox" and "computer vision toolbox"?
loss functions for time series prediction:
The current deep learning toolbox is missing loss functions for time series prediction, e.g. "DTW, Soft-DTW, DILATE" loss, and although the signal processing toolbox has a dtw function, it does not support dlarray type data input?
Video reading and processing
The latest version of R2021a only supports fileDatastore/videoReader etc. to load various video datasets, and the read IO bottleneck is very inefficient and lacks a decord-like library for efficient video processing
Bayesian Deep Learning Convolution Network(BDL),Bayesian Neural Network(BNN)
Binary Neural Networks
Does current matlab 2021a deeplearning toolbox support Binary Deep Neural Networks (BNN)? How is it trained and compressed? If not, will future versions integrate this feature? thanks
reference:
@Adam Danz,Thank you for your useful advice！This is another indication that matlab is very good, not the latest, but the most robust and reliable algorithm integration!
@cui your suggestions are imaginative, specific, and I'm sure it's valuable to the company to see what users would like. It would be even more helpful to include a comparison to existing features that come close to achieving your suggestions. I'm not an employee of MathWorks nor am I speaking for them, this is just my opinion.
For example, there are already object tracking algorithms available in Matlab (see computer vision toolbox). At the current pace of technology, algorithms are created much faster than the pace of development, especially for companies like MathWorks that must thoroughly test for backward compatibility and redundancy. This is where the File Exchange often comes in handy to fill in the gaps. By listing every possible missing algorithm, the SNR of focal points for future releases becomes weaker. That could be strengthened by including arguments for why a new feature is needed rather than just listing missing features.
As far as I can tell, there does not seem to be very much in the ways of documentation for Excel-based work (i.e creating and manipulating charts). I'd love to see some of this material added into the Documentation search if at all possible.
OK, thanks for the hint -- I had thought that
borders=get(theCell, 'Borders');
theBorder=get(borders, 'Item', border);
set(theBorder,'LineStyle', style,'Weight',weight);
would have returned the wanted border Item object, but apparently not...I still don't fully understand that, but the following does work....
theBorder=borders.Item(whichBorder);
set(theBorder,'LineStyle', style,'Weight',weight);
although one apparently can't write
set(borders.Item(whichBorder),'LineStyle', style,'Weight',weight);
in typical MATLAB syntax fashion.
function SetBorder(sheetReference, ranges, whichBorder, weight, style)
if style==XlLineStyle.xlDouble, weight=XlBorderWeight.xlThick; end % only combination that works
if ischar(ranges), ranges=cellstr(ranges); end
if isenum(whichBorder), whichBorder=int32(whichBorder); end
if isenum(weight), weight=int32(weight); end
if isenum(style), style=int32(style); end
for i=1:numel(ranges)
try
range=sheetReference.Range(ranges{i});
borders=get(range, 'Borders');
theBorder=borders.Item(whichBorder);
set(theBorder,'LineStyle', style,'Weight',weight);
catch ME
fprintf('Error in function SetBorder.\nError Message:\n%s\n', ME.message)
%warning('Error in function SetBorder.\n%s', ME.message)
end
end
end
Experimentation also revealed that one can't use any combination of the enumerations (which limitation is also NOT documented). Only with the "Thick" weight is a double-line rendered; all the others are a single line of medium weight.
All kinds of things aren't documented and left for only trial and error to uncover...
The above could be cleaned up and forget about trying to use enumerations, but it's expedient for the moment and there are far more important issues to fix rather than continuing to throw time down the rat hole...
Thanks, you did get me pointed to what can be made to work--muchly appreciated!!
So whassup w/
% 9 - xlEdgeBottom Border at the bottom of the range
in the doc enumerations and nothing listed for 1 thru 4???
This still makes no sense to me whatsoever...
I'm in the middle of another debug session at moment so will check again but I tried a loop of 1:N before and still had no joy I'm pretty sure...
Thanks for the feedback; will report.
borders.Item(1).LineStyle = -4119 will give you double line on one border, I think the right one. Bottom border is one of Item between 2 and 4 so check those.
I'm quite sure I won't be able to offer any help. I was just curious. But I feel your pain having dabbled with doing much simpler mucking around in PowerPoint from Matlab.
Oh, here's the generic routine I tried to write, taken from Image Analyst's utilities package that illustrated some specific choices rather than being general. You'll see the kludge to get around the MATLAB enumeration problem--I had built those classes as a user aid to be able to see the constants instead of having to look them up or have in comments or bury magic constants in source code. That proved to be, so far, an almost pointless exercise, too.
function SetBorder(sheetReference, ranges, border, weight, style)
% borders is a collections of all cell borders. Must set each.
%
% my_border = get(borders, 'Item', XlBordersIndex);
% set(my_border, 'ColorIndex', 3);
% set(my_border, 'LineStyle', 9);
%
% Excel XlBordersIndex Enumeration
% 5 - xlDiagonalDown UL diagonal down to LR
% 6 - xlDiagonalUp LL diagonal up to UR
% 7 - xlEdgeLeft Border at the left edge of the range.leftmost only
% 8 - xlEdgeTop Border at the top of the range.
% 9 - xlEdgeBottom Border at the bottom of the range.
% 10 - xlEdgeRight Border at the right edge of the range.
% 11 - xlInsideVertical Vertical borders for all the cells in the range except borders on the outside of the range.
% 12 - xlInsideHorizontal Horizontal borders for all cells in the range except borders on the outside of the range.
%
% Excel XlBorderWeight Enumeration
% Specifies the weight of the border around a range.
% Name Value Description
% xlHairline 1 Hairline (thinnest border).
% xlMedium -4138 Medium.
% xlThick 4 Thick (widest border).
% xlThin 2 Thin.
%
% Excel XlLineStyle Enumeration
% Specifies the line style for the border.
% Name Value Description
% xlContinuous 1 Continuous line.
% xlDash -4115 Dashed line.
% xlDashDot 4 Alternating dashes and dots.
% xlDashDotDot 5 Dash followed by two dots.
% xlDot -4118 Dotted line.
% xlDouble -4119 Double line.
% xlLineStyleNone -4142 No line. (Alias xlNone)
% xlSlantDashDot 13 Slanted dashes.
if ischar(ranges), ranges=cellstr(ranges); end
if isenum(border), border=int32(border); end
if isenum(weight), weight=int32(weight); end
if isenum(style), style=int32(style); end
for i=1:numel(ranges)
try
theCell=sheetReference.Range(ranges{i});
borders=get(theCell, 'Borders');
thisBorder=get(borders, 'Item', border);
set(thisBorder,'LineStyle', style,'Weight',weight);
catch ME
fprintf('Error in function SetBorder.\nError Message:\n%s\n', ME.message)
%warning('Error in function SetBorder.\n%s', ME.message)
end
end
end
It also has the aforementioned bizarre behavior that the warning line is totally ineffective -- I just thought about the fact this routine is still a static method in a class having kept the organization IA had posted. Haven't tested whether moving it into its own m-file would fix that problem--good guess it might I think. Although I couldn't find anything to that end in the MATLAB doc, either.
Paul/Mario -- thanks for taking an interest... :) I really didn't expect anybody to delve into this; I was just frustrated and ranting...
Here's the little testing routine I wrote thinking if I manually set some borders in a spreadsheet I could then retrieve what was in the cells and use that to figure out what should write -- needless, to say, I left more confused than I came--
function GetBorderInfo(sheetReference, ranges)
% return borders collection info for range given
if ischar(ranges), ranges=cellstr(ranges); end
for i=1:numel(ranges)
try
theCell=sheetReference.Range(ranges{i});
borders=get(theCell, 'Borders');
nBorders=boarders.Count;
thisBorder=get(borders, 'Item', border);
set(thisBorder,'LineStyle', style,'Weight',weight);
catch ME
fprintf('Error in function SetBorder.\nError Message:\n%s\n', ME.message)
%warning('Error in function SetBorder.\n%s', ME.message)
end
end
end
I opened the ActiveX session first, then used
GetBorderInfo(wksht,'D2')
and then poked around in the debugger trying to figure out what was what.
I did recognize the problem line so set the breakpoint at the beginning of the
thisBorder=get(borders, 'Item', border);
line and then did all my poking around in the debugger. I just got so frustrated I never tried to fix the code but just gave it up as lost cause and wasted day.
If you look at the doc for enumerations
the values are
Name Value Description
xlContinuous 1 Continuous line.
xlDash -4115 Dashed line.
xlDashDot 4 Alternating dashes and dots.
xlDashDotDot 5 Dash followed by two dots.
xlDot -4118 Dotted line.
xlDouble -4119 Double line.
xlLineStyleNone -4142 No line.
xlSlantDashDot 13 Slanted dashes.
but any value I tried to write other than 1 or -4142 returned the error and didn't set the line style desired, but I did, in fact see the value -4119 returned from poking around after I had manually set and save the cell border in cell D2 of the test workbook.
I guess the thing about " If you set borders.LineStyle to some value, 1-4 are set" explains why the enumerations begin at 5 although if that is documented anywhere, I've not been able to find it, nor do any examples mention the fact.
So, what I wanted was a bottom double line of either thin or medium width...but tried to write a generic routine to be able to set any as desired.
It worked/works for a continuous line, but nothing else...gets what appears a random selection of continusous or various dotted/dashed, etc., but never a double to be seen no matter what I tried to write.
It's also not documented what the expected type of the values passed is to be; afaict by symptoms, MATLAB set() converts any native numeric to whatever internally although as the other Q? I posted notes (to which there's been no response so far) the MATLAB enumeration type seems to actually pass some object, not the equivalent value as value as one would expect an enumeration should/would do. That's just rude!!!
Anyways, any further light you can shed would be greatly appreciated, but I remain in the camp believing that writing ActiveX to interact with Excel is worse than having root canal without anesthesia...
Where in your code is the variable borders defined, as used in this line?
nBorders=borders.Count
Where did you see this line?
thisBorder=get(borders, 'Item', border);
I hope I understood what you wanted to do.
This line is not good.
thisBorder=get(borders, 'Item', border);
Try this
theCell=sheetReference.Range(ranges{i});
borders=get(theCell, 'Borders'); % borders return the borders that are bounded by range Items(1:4), where 5:6 are diagonals
set(borders,'LineStyle', style,'Weight',weight);
There are some weird things of course which are misisng from the documentation. borders.Count returns 6, borders.Items go to 12 (probably when selection is 2D). If you set borders.LineStyle to some value, 1-4 are set.
Maybe you've found " their methods and properties are described meaningfully and it can be easy to narrow down the search" to be the case, but my experience has not been so fortunate. Maybe you have enough familiarity with the object model that you can figure out how to make a Range or Cell or whatever reference to whatever it is you're looking for work, but I'm continually fight the battle of just how to get to the property of interest when it's in some collection of somethings underneath the cell or range or whatever...
I spent all day yesterday in pursuit of how to set borders programmatically and came up empty -- stuff that is documented just doesn't work or I can't figure out how to use it--another frustration I discovered raised the following Q?
and the follow-ons to the template code I used as starting point at
https://www.mathworks.com/matlabcentral/answers/585908-excel-formatting-using-matlab#comment_1554985
If one tries to use the Xl enumerated constants to set the border line styles, Excel barfs...but if you set a given border interactively and then try retrieve the value of the constant from the saved workbook, it looks to be that same value. Whassup with that!!!??? I finally gave up in frustration as has been my luck about as much as not for anything outside just writing data.
If you can make heads nor tails of the Borders collection documentation, you're a better maven than I...I have no klew from it how it is constructed and what it means to get a Count == 6 when the doc says it is a collection of four??? And the enumerations go to 15 or so??? Makes zero sense to me.
>> excel=actxserver('Excel.Application');
>> wbk=excel.Workbooks.Open(fullfile(cd,'Testing.xlsx'));
>> wksht=Excel_utils.sheetReference(excel,1)
wksht =
Interface.000208D8_0000_0000_C000_000000000046
K>> get(sheetReference.Range("A1").Borders,'LineStyle')
ans =
NaN
K>> get(sheetReference.Range("D2").Borders,'LineStyle')
ans =
NaN
K>> nBorders=borders.Count
nBorders =
6
K>> get(borders)
Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
Creator: 'xlCreatorCode'
Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
Color: 0
ColorIndex: -4142
Count: 6
LineStyle: NaN
Value: NaN
Weight: NaN
ThemeColor: NaN
TintAndShade: NaN
K>>
K>> borders
borders =
Interface.00020855_0000_0000_C000_000000000046
K>> borders.Count
ans =
6
K>> borders(1)
ans =
Interface.00020855_0000_0000_C000_000000000046
K>> borders(2)
Index exceeds the number of array elements (1).
K>>
% OK, so it's not an array of borders of nCount
K>> for i=1:nBorders,get(b,'item',i),end
ans =
Interface.00020854_0000_0000_C000_000000000046
ans =
Interface.00020854_0000_0000_C000_000000000046
ans =
Interface.00020854_0000_0000_C000_000000000046
ans =
Interface.00020854_0000_0000_C000_000000000046
ans =
Interface.00020854_0000_0000_C000_000000000046
ans =
Interface.00020854_0000_0000_C000_000000000046
K>>
% OK, so whatever an Item is, there are six of those???
K>> for i=1:nBorders,b=get(borders,'item',i),get(b),end
b =
Interface.00020854_0000_0000_C000_000000000046
Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
Creator: 'xlCreatorCode'
Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
Color: 0
ColorIndex: -4142
LineStyle: -4142
Weight: 2
ThemeColor: NaN
TintAndShade: NaN
b =
Interface.00020854_0000_0000_C000_000000000046
Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
Creator: 'xlCreatorCode'
Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
Color: 0
ColorIndex: -4142
LineStyle: -4142
Weight: 2
ThemeColor: NaN
TintAndShade: NaN
b =
Interface.00020854_0000_0000_C000_000000000046
Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
Creator: 'xlCreatorCode'
Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
Color: 0
ColorIndex: -4105
LineStyle: 4
Weight: -4138
ThemeColor: 'Invoke Error, Dispatch Exception: The parameter is incorrect.←↵'
TintAndShade: 0
b =
Interface.00020854_0000_0000_C000_000000000046
Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
Creator: 'xlCreatorCode'
Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
Color: 0
ColorIndex: -4105
LineStyle: -4119
Weight: 4
ThemeColor: 'Invoke Error, Dispatch Exception: The parameter is incorrect.←↵'
TintAndShade: 0
b =
Interface.00020854_0000_0000_C000_000000000046
Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
Creator: 'xlCreatorCode'
Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
Color: 0
ColorIndex: -4142
LineStyle: -4142
Weight: 2
ThemeColor: NaN
TintAndShade: NaN
b =
Interface.00020854_0000_0000_C000_000000000046
Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
Creator: 'xlCreatorCode'
Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
Color: 0
ColorIndex: -4142
LineStyle: -4142
Weight: 2
ThemeColor: NaN
TintAndShade: NaN
K>>
% But some show an error, but no indication of which parameter it thinks is
% incorrect...
% The Xlborder enumeration for bottom line is 9, though, which is >6, but
% it also returns something that is, the value I tried to write of -4119
% which is supposed to be a double line by the enumerations values in the
% documentation???
K>> b=get(borders,'item',9)
b =
Interface.00020854_0000_0000_C000_000000000046
K>> get(b)
Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
Creator: 'xlCreatorCode'
Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
Color: 0
ColorIndex: -4105
LineStyle: -4119
Weight: 4
ThemeColor: 'Invoke Error, Dispatch Exception: The parameter is incorrect.←↵'
TintAndShade: 0
K>> set(b)
ans =
struct with fields:
Application: {}
Creator: {'xlCreatorCode'}
Parent: {}
Color: {}
ColorIndex: {}
LineStyle: {}
Weight: {}
ThemeColor: {}
TintAndShade: {}
K>>
K>> get(b)
Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
Creator: 'xlCreatorCode'
Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
Color: 0
ColorIndex: -4105
LineStyle: 1
Weight: -4138
ThemeColor: 'Invoke Error, Dispatch Exception: The parameter is incorrect.←↵'
TintAndShade: 0
K>> set(b,'ThemeColor',1)
K>> get(b)
Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
Creator: 'xlCreatorCode'
Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
Color: 16777215
ColorIndex: 2
LineStyle: 1
Weight: -4138
ThemeColor: 1
TintAndShade: 0
K>> set(b,'LineStyle',-4119)
K>> get(b)
Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
Creator: 'xlCreatorCode'
Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
Color: 16777215
ColorIndex: 2
LineStyle: -4119
Weight: 4
ThemeColor: 1
TintAndShade: 0
K>> b=get(borders,'item',5)
b =
Interface.00020854_0000_0000_C000_000000000046
K>> get(b)
Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
Creator: 'xlCreatorCode'
Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
Color: 0
ColorIndex: -4142
LineStyle: -4142
Weight: 2
ThemeColor: NaN
TintAndShade: NaN
K>> set(b,'LineStyle',-4119)
K>> get(b)
Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
Creator: 'xlCreatorCode'
Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
Color: 0
ColorIndex: -4105
LineStyle: -4119
Weight: 4
ThemeColor: 'Invoke Error, Dispatch Exception: The parameter is incorrect.←↵'
TintAndShade: 0
K>>
K>> sheetReference.Parent
ans =
Interface.000208DA_0000_0000_C000_000000000046
K>> wbk=ans;
K>> wbk.Name
ans =
'Testing.xlsx'
K>> wbk.Save
K>> wbk.Close
>> delete(excel)
>> clear('excel')
Part of extended debug session trying to figure out how to set a double line -- no success whatever; any of the values in the enumeration other than continuous (3) or none (-4142) error but if read a cell which manually set to have the desired linestyle, it returns the same value.
I've no explanation for why nor found any way in which can make the desired change programmatically.
One can't write VBA expressions that do assignments in ActiveX syntax nor are the values actually passed for the enumerated constants visible in how they're passed other than as the decimal values shown in the documentation, so if they don't work, one's pretty-much at an impasse.
That's been my frustration on almost everything I've ever tried -- it simply is not a 1:1 translation; eventually one usually can find someone on one of the Excel newsgroups or code search that has stumbled over same/similar problem and find the magic incarnation, but I've not found anything yet to get around this one.
Sorry for the rant...but was pretty-much a lost day. :(
Sadly, my hammer isn't good enough to understand these xkcd comics xD
If you look at complete object model, then yes, but we are mostly interacting with one (or few) objects so one needs to look at their documentation. Luckily their methods and properties are described meaningfully and it can be easy to narrow down the search. What would be cool is to be able to track the existing methods, properties and their values in the variable editor, like uiinspect does with the java components.
_" you can find some documentation on Excel here..."
:) Indeed.
That's the problem--there's far too much documentation; the Object Model is so complex one soon has spent the entire day trying to figure out one little piece; particularly if new to trying to use ActiveX.
And, of course, VBA syntax doesn't directly translate to ActiveX, either...
I don't disagree that it "isn't TMW's job!" but it is a bugger to do anything at all quickly unless already completely familiar with and adept in the object model.
I wonder if there's a team that focuses on Excel integration.
I don't think this would ever happen, if you can interface with COM object then you can operate it with the methods that have been implemented by the program developer. It is on us to use that with whichever language we want to. I am glad that functions like writecell, readcell and others related to these exist, as I think it's hard to interface with Excel, programmatically through COM object, so thanks TMW for that.
@Robert Guldi I'll take a look at question you posted tomorrow. Meanwhile, you can find some documentation on Excel here https://docs.microsoft.com/en-us/office/vba/api/overview/excel/object-model
I'm working specifically with ActX server, and I eventually figured out everything I needed to get my program to work. While it is certianly easy to plot things in Matlab, I needed the data (which started out as a TDMS file) to be easily manipulated and viewed by people who have are very comfortable with Excel and want to be able to view and edit the charts and data with ease, much like dpb was talking about. MATLAB is very powerful and it can process the chart changes with hundreds of thousands of datapoints each much faster than Excel can, as it has to regenerate the chats any time there is a change, and it's reduce the total time to process by a factor of 10, and I think any better documentation would greatly help a very large number of people.
Now I want one.
"My first programming was done in the mid 90s with TI Calculators using..."
Don't get me going on geezer stories or I'll be reminiscing about Wang nixie tubes and even further back... :)
I wasn't aware that Excel charts can be mainpulated with Matlab or that anyone would even want to do this but I realize there are lots of different workflows outside of my own. I don't understand how it would be advantagous to use Matlab, a much more powerful tool than Excel, to create charts in Excel. The skill-set needed to do that seems greater than the skill-set needed to create a plot directly in Matlab. But you bring up a good point about programmers forced to make things work for people using Excel.
Manipulating sections of a spreadsheet or workbook is understandable and since Matlab does have functions that can read/write to specific sections of worksheet, I didn't realize there was anything more that was needed.
My first programming was done in the mid 90s with TI Calculators using a program that could upload code written in an IDE to the calculator. Then I started programming macros in Excel for years. But as soon as I learned Matlab, the rest was history concerning Excel/TI.
Matlab documentation on Python integration is growing. I wonder if there's a team that focuses on Excel integration.
Robert specifically mentioned creating and manipulating charts -- and he's right, there are no builtin facilities for handling more than reading/writing data to/from Excel.
I don't know/think that it is TMW's place to be writing MS Excel documentation translation to/from Excel, but there are a lot of people apparently trying to manipulate objects such as charts inside Excel documents from/with MATLAB.
I've just been through the process of simply trying to format an automated template created from MATLAB code and again, while it's almost trivial to build the data and formulas and write them with writecell or writetable (and, if one looks, one discovers all writecell does is call cell2table and then pass the result to writetable), but then one has to reopen the file with an Excel COM object and mung in the object model to do anything else like set number formatting.
Or, I discovered that if one needs to construct the spreadsheet piecemeal (say doing an agglomeration of data across a large collection by the unique IDs present so one gets those at one time instead of all at once) that even with writetable, the overhead of opening/closing the file is, at best, painfully slow for more than just one or two sections, or at worst, completely hangs the system.
To accomplish this goal, one has to revert to a hack of opening a COM object and writing all the data before then saving and closing the file. At least with xlswrite one has the outline of a code to do that that some others have already extended on FileExchange to do that.
Doing so reduced a process from over an hour with writetable when it did actually finally(!) run to only 7-10 seconds.
While I'd be happy if I never had seen Excel for the most part, there are millions using it and I'm stuck as a volunteer for local nonprofit with trying to make some of their manual processes built around a plethora of spreadsheets at least somewhat more productive by packaging applications to process these data for them.
Fortunately, I haven't had to deal with Excel charts (at least so far, althoug that might be on the horizon), just the data that is much simpler to pull from the dozens of workbooks, assimilate with MATLAB tables and vector operations with rowfun and friends and then put back as compared to trying to write the same functionality in VBA. But, the office folks still have to use Excel so can't just throw it away -- all a long-winded way to say I can relate to the wish and understand how it might be a need, not just a preference instead of plotting in MATLAB (and, truthfully, for some kinds of management data, the abilities packaged in Excel are light years ahead of TMW/MATLAB in functionality and appearance).
Are you referring to tables or timetables, readtable, readcell, readmatrix, readtimetable, textscan | writetable or perhaps some of the support functions such as readvars, setvaropts,detectImportOptions, Import Tool, preview? Also see
One thing would be: defining classes in notebooks (Live Scripts in MATLAB case). That's a normal thing for years now in Python, for example.
You might be interested in https://www.mathworks.com/products/reference-architectures/jupyter.html which is new since you posted your wish.
Deep Learning toolbox:
groupedConvolution2dLayer since R2019a support, but groupedConvolution3dLayer don't support ??? (until R2020b)
update:
R2022b still can't support
Mathworks-China does not actively care about my enhancement needs, but is interested in my work unit and personal contact number. I am very disappointed!
Maybe you could merge this answer with your previous answer with deep learning toolbox suggestions? I would also encourage you to contact support with these suggestions. It seems you have a lot of them, so they might be interested in hearing from you.
Deep Learning toolbox:
- action recognition: mmaction2
- video understanding, X3D: https://github.com/facebookresearch/SlowFast ，https://github.com/kenshohara/3D-ResNets-PyTorch
- reID: https://github.com/JDAI-CV/fast-reid
- track: https://github.com/ZQPei/deep_sort_pytorch，https://votchallenge.net/challenges.html
- Metric Learning: https://github.com/KevinMusgrave/pytorch-metric-learning
- image Classification: Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition
- object detection: Centernet, https://github.com/xingyizhou/CenterNet , efficientDet, https://github.com/google/automl/tree/master/efficientdet，detectron2，https://github.com/facebookresearch/detectron2 ,PointRCNN,https://github.com/sshaoshuai/PointRCNN
- face recognition: insightface, https://github.com/deepinsight/insightface, sphereface,https://github.com/wy1iu/sphereface, arcface,https://github.com/ronghuaiyang/arcface-pytorch
- Semantic segmentation, instance segmentation，https://github.com/facebookresearch/maskrcnn-benchmark , Semantic Segmentation (Most of them are difficult to implement with matlab)
- deeplearning preprocess data: https://github.com/NVIDIA/DALI
- StyleGAN: https://github.com/NVlabs/stylegan
- DeepNetwork Architecture：Mnasnet，https://github.com/tensorflow/tpu/tree/master/models/official/mnasnet , Bayesian-Neural-Networks , Graph Convolutional Networks
- similarity search: https://github.com/facebookresearch/faiss
- deepfakes: https://github.com/deepfakes/faceswap
- deepnetwork inference performance: https://github.com/microsoft/onnxruntime ,https://github.com/zuoqing1988/ZQCNN
- image Labeler app: https://www.mathworks.com/matlabcentral/answers/789159-how-does-image-labeler-app-realize-this-interactive-operation-by-writing-a-program?s_tid=srchtitle
and so on....
The above are influential applications of deep learning in various aspects, but it is difficult to reproduce in matlab. Although Matlab2019b version supports automatic differentiation mechanism, it is still difficult to implement algorithms in matlab. The efficiency of the differentiation mechanism is not high, and many operators do not support it. I tried to implement the more famous yolov3/v4 algorithm with the latest MATLAB2020a version, but it is still not satisfactory
In summary, my personal suggestions are like my personal answer above, and I hope that future versions can improve a lot!
以上都为深度学习在各个方面有影响力的应用，但是在matlab中复现困难，虽然Matlab2019b版本支持自动微分机制，但仍然不易在matlab实现算法，微分机制效率不高，很多operators也不支持。。。
总之，我的个人建议就像上面的个人回答建议一样，我希望将来的版本可以有所改善！
I have been following the latest development progress of "deeplearning toolbox", and although there are many new features listed, most of them are superficial improvements with very limited substantive support for the "operator", which is still very inadequate for experienced and in-depth researchers.
I hope that mathworks will seriously develop basic modules for the toolbox, instead of small features that are not very important to attract beginners.
我一直在关注“deeplearning toolbox”的最新开发进展，虽然有很多新的功能项被列出来，但这些大部分都是表面上改进，实质上的支持operator非常有限，对于有经验的深入研究者来说还显现的非常不足。
MATLAB
I very much hope that the official version will strengthen the readstruct function in the future! Lack of complete uniformity to support more format requirements.
Deep Learning toolbox:
How to visualize the changes in dlarray/weight distribution with histograms in deep learning?
Histogram displays how the trend of tensor (weight, bias, gradient, etc.) changes during the training process in the form of histogram. Developers can adjust the model structures accurately by having an in-depth understanding of the effect of each layer.
it's lucky to hear that recently matlab version support deepnetwork visualize weight distribution. here is one example link
@Adam Danz Thank you for your suggestions, I look forward to the official future version will be greatly improved！
The first plot is similar to Matlab's waterfall plotting function.
The GIF is too quick for me to make any sense out of it.
Perhaps you could directly suggest these ideas using Contact Us - MATLAB & Simulink
This thread is not an official enhancement request.
To me it's simply a place where people come to emplty the bag of complains.
TMW will not comment in open forum on pending development or toolbox direction; you can be confident they have seen and read the above, but you cannot interpret no response as no interest.
If you wanted more direct input, submit items as enhancement requests, but again, TMW simply does not share its plans in open forums.
Related issues：
It seems that Mathworks is not interested in many of the improvement questions (above and below) raised by me, and I regret that there is still no answer!
No big deal but I wish SIGN(X) function could return 1 for X=0 instead of 0. May be implementing an option to not break the compatibility.
I just rarely use SIGN because of this exception choice.
I’m not up on IEEE 754 standards obviously, so signed zeros are new.
"Perhaps I’m missing something, however if 0 can never be either positive or negative"
You might be missing the fact that IEEE 754 has signed zeros. Signed zeros explain why this happens:
x = -0;
1/x
Bruno Luong and I would like a function that gives the sign of that zero before we do a calculation with it.
Sometimes I work with zeros and infinities as values, so this is actually information that I have needed to use. Currently I have to perform an operation on the value to determine the sign, or do some bit-bashing and scale the results of that (much as Bruno Luong showed). The goal is that sign.
Another way to look for a logic is that the normsign(z) is the formula
s = exp(1i*angle(z))
or
s = exp(1i*atan2(imag(z),real(z)))
z is real or complex.
There are a lot of math formula that can be wrtten down very compact and simplified implementation with those convention (normsign), such as recursive formula for derivativex of Zernike functions, etc...
I can't think I ever really need TMW sign() convention. The only nice property is actually it's a limits of a smooth series of functions.
Perhaps I’m missing something, however if 0 can never be either positive or negative, I don’t understand the rationale of giving it a sign here. Obviously the sign of -eps (or -realmin) is -1 and of eps (or realmin) is +1 however those are not actually 0.
.
I correct m code, the first value for NaN input should be NaN (arguably need)
For the record (because I couldn't immediatly see what the effect would be):
normsign([nan, -inf, -1, -0, 0, 1, inf])
I would implement the sign like this
A = [nan, -inf, -1, -0, 0, 1, inf]
normsign(A)
function s = normsign(A)
b = typecast(double(A),'uint8');
s = 1-2*(bitand(b(8:8:end),0x80)>0);
s(isnan(A)) = NaN; % arguably need
s = reshape(s,size(A));
end
Could you remind me what output you are hoping for, for each of these values:
A = [nan, -inf, -1, -0, 0, 1, inf]
sign(A)
"You might have needed access to the "sign bit", but that is different than the sign."
No, I did not need to "access" the sign bit, I needed to the sign of the value, i.e. the information that those bits represent, but I certainly did not want not the bits themselves.
I did not confuse information with how information is encoded.
You might have needed access to the "sign bit", but that is different than the sign.
Getting to the sign bit of double precision numbers is not difficult:
A = [-100/3, 0, 42.1234]
bitget(typecast(A, 'uint64'), 64)
For single precision, extract bit 32.
I do not recall ever having encountered a programming language in which the "sign" operator returned the "sign bit" instead of the signum value.
FORTRAN's SIGN functions are "transfer of sign", and so avoid the issue of representation. However, SIGN(1,0) would be 1 not 0.
ALGOL 60's SIGN function was defined in terms of -1, 0, +1.
LISP's signum function was defined in terms of -1, 0, +1.
"Making sign(0) = 1 would disturb a number of the mathematical properties."
It already disturbs properties of the binary floating point numbers.
Why do some "mathematical properties" have a higher importance than binary floating point properties? I have on occasion required the sign of a binary floating point number (including zeros): are my usecases unimportant?
Perhaps two functions would resolve this:
SIGN (mathematical, consistent with current function)
SIGNF (of floating point number stored in memory)
Personally, I do not believe that sign(0) = 1 is reasonable. sign() is signum which is defined as 0 for 0.
Making sign(0) = 1 would disturb a number of the mathematical properties.
Rather than getting it to return 1, why not simply (and correctly) return the sign of the zero?:
Why I prefer sign(0) = 1?
To me y = sign(x) should satisfy these two important properties
norm(y) = 1
x = y * abs(x)
This uniquely determines y for any x ~= 0 (real, complex or even quaternion). For x=0, any y=exp(1i*alpha) with alpha real meet both properties, so just select it as y=1 for arbitrary convention.
However the MATLAB SIGN() function does NOT satisfy the first property for x=0.
Adams's trick sign(x+realmin) is interesting, but it just shift the issue to x=-realmin.
It is common for people to write terms in the form
something + value * u[x-offset] %unit step function
and common for people to implement that either as
something + value * heaviside(x-offset) %OR
something + value .* (x>= offset)
and occasionally as
something + value .* (sign(x-offset)+1)/2
except wanting sign() to return 1 at 0 to achieve the same boundary condition.
And that is fine provided that value is finite when it is unselected, with non-finite being a problem because 0*inf -> nan, 0*-inf -> nan, 0*nan -> nan.
This is not a problem that can be cured by using the heaviside() or sign() functions or the >= operators... it needs something more like piecewise:
something + piecewise(x >= offset, value, 0)
It is not a problem in theoretical mathematics because Heaviside is more a distribution than a function (just like Dirac Delta is a distribution) and Limit of the distribution acts to remove the problem.
sign() returns the sign of -inf/+inf (-1/1) and returns NaN for NaN inputs.
Wanting sign to return 1 at 0 would seem to be for use as Heaviside function.
Heaviside is available in the Symbolic Toolbox as heaviside() -- though you would want to sympref('heavisideatorigin', 1).
For numeric work it can often be replaced by (x>=0)... with the usual warnings that if what is being multiplied is nan or inf then you get nan out... but that would happen for sign() as well.
In such cases I've used, sign(x+realmin)
Example,
sign((-2:2)+realmin)
When typing code, if user wants to reference a variable or a function, one could click tab and get a matching list of functions and variables. Would it be useful to split this functionality with modifier if user wants to reference one of the two? As an example Shift +Tab for variables and Tab for functions and variables (not to break the current functionality).
Implement equivalent to C inline-function (or macro) so as calling this function on small data won't be penalized in speed with the over-head.
Implement family of (numerical) data structure with O(1) inserttion, removal, that includes chain-list, binary tree, Fibonachi tree, red-black tree etc, etc, ... the performance must be focus point. Don't care if they are encapsulated or not in the OP, just don't make that reduce the performace.
The following changes to the internal volume renderer would make Matlab much more useful for volume visualization:
- True RGB volume visualization (vs. just scalar data + colormap)
- Independent, decoupled specification of alpha values and intensity values
- The ability to combine volume images with standard axes and axes objects like points, lines, patches, etc.
And add the ability to pass axes scales, similar to the image() XData property, and maybe even support irregularly spaced points
Ok, this is pretty minor, but:
For common functions that return values as the first output and indices as the second, it would be nice if there was a direct way of getting the indices, so I could do things like this:
x = [1 2 5 4 5];
y = 1:5;
y_xsorted = y(sortInd(x));
% or
y_xunique = y(unique(x,'ia'));
instead of needing to create temporary variables:
x = [1 2 5 4 3];
y = 1:5;
[~,idx] = sort(x);
y_xsorted = y(idx);
[~,idx] = unique(x);
y_xunique = y(idx);
I know I could make wrappers myself, but this seems like a case where a builtin function could potentially be noticeably optimized. (And, is more straightforward than rebuilding the whole output system)
A "select" clause for readtable() and kin.
For example one user only wanted to read the rows in which one particular variable had a particular value
It could at some point be implemented in terms of a rowfun() type function that got passed the variables for a row and could make arbitrary decisions based upon the row contents.
However an earlier stage could potentially have pairs of variable names (or numbers) and a vector or cell array of values, in which the selection code did an ismember(). Such a facility could be further improved if there were a "sorted" option (so the code could figure out when to give up looking -- any one value should no longer be looked for if a larger value were encountered). Or even "grouped", which would not imply sorted as such but would imply that when you find a value that all the instances of that value that exist will be in adjacent rows and so as soon as you detect a change you can know to give up looking for that value.
A "soft interrupt" -- that is, an interrupt that can be handled with try/catch .
For example I am running a long calculation at the moment that iterates several times, and each iteration can take more than an hour. I don't always have the patience to wait through as many iterations as I asked for. For my purposes, the output of the last successful iteration would be "good enough" if I were to ask to interrupt the function.
I can control-C, but that interrupts the function completely, losing all of the outputs, and losing the function variables.
If I could somehow "soft interrupt" and have it return the current values of all variables, then that would be good enough for my current task. But the generalization of that would be the ability to catch a soft interrupt in a try/catch so that the code could make whatever final summariziation it needed to in order to create usable output variables. Furthermore, even though my own routine might be happy to return the "last good" values of the output variables, any soft-interrupt I requested might well get received while some lower-level routine had control that did not know about soft-interrupts, so I would want interrupt (with no return values) to apply to those layers until control reached my handler for a clean run-down.
My solution in a similar case was to check whether a certain file ('FAIL_MODE.txt') exists in the directory at the end of each iteration. If it does, break the loop. Then, to stop the program, all I had to do was create the file (which is pretty trivial unless Matlab has frozen the whole computer)
onCleanup would have to either save() or store into global variables. The point is to be able to have a smooth return of values.
Outputs not assigned would still generate the same error messages as at present.
Function interrupted at unknown state is no worse of a problem than functions that error and are caught by try/catch at present.
It is common for programming systems to have a control-C handler .
Just program an onCleanup to save whatever that needs to be saved. Ctrl-C do the rest.
What you propose is a dream but how to deal with variables/outputs not assigned and function interrupted at unknown state?
Only the programmer can know his workflow. So no possible to implement this lazy feature IMO.
a function that would need a manual for a GUI to be feasible
That's probably the same program being referred to ;-)
Another solution in current Matlab would be a GUI with a flag controlled by a button. But I believe you have previously talked about a function that would need a manual for a GUI to be feasible.
That last section sounds exactly like try-catch, so essentially you want to be able to trigger an error, which will then cascade up to your softtry-catch block. Either way this sounds like a useful feature.
It can be a pain to dig into the program each time, and requires detailed knowledge of the program.
... I've already held off several releasing this program because I am not happy with the command line interface I have for it (too many obscure options that are rarely (but sometimes) useful)... I don't expect people to manually poke through the code to put in breakpoints and know which variables to save... but it would be common for people to get tired of the program executing for hours and to hope not to lose the progress so-far if they interrupted. Yeah, I could save to global variables, but...
ICK
This isn't exactly what you're asking for but you could pause your running program (which may pause in a lower-level routine.) Then select your function's workspace and save the variables you want to keep. Or if you want MATLAB to continue the current iteration and stop just before it starts the next, add a breakpoint on the line where the next iteration will start and continue the current iteration. This functionality has been part of MATLAB since release R2016a.
Editor won't restrict a substituion to selected area...best it knows of is the function. Uncool in the max! I'm now having to fix up a bunch of stuff shouldn't have to have done... :(
I know I've said it before, but it's still missing and still important:
We need a way to gather all of the outputs of function call into a cell in the middle of an expression .
I know this may be tricky to implement internally. There are internal rules that are hard to work out, that have to do with how many outputs to request from functions. For example,
[A, B] = cellfun(@C, D, 'uniform', 0)
somehow passes knowledge of "two outputs" to C -- for example if you were to use @max then A would be a cell array of the maxima and B would be a cell array of the indices. The situation can get more complicated than this, and figuring out all the cases can make your head hurt. But we do know that any expression C(D(E)) that D(E) will be evaluated asking for exactly one output and that would be passed into C... but the knowledge of multiple outputs would be passed to C and yet not D.
The number of outputs to use is not inherently clear. If for example you call ode45 and ask to gather the outputs, are you asking for the common TOUT, YOUT case, or the full TOUT,YOUT,TE,YE,IE ? There are some cases where extra outputs can be expensive to calculate, so even though an operation that gathered "maximum" outputs might be useful, it is not always desireable, so ability to select the number would be useful.
Then there are issues with, for example, deal(), where you can have any number of outputs with just a single input:
[A,B,C,D] = deal(123)
would initialize A, B, C, and D all to 123. So if you ask to gather "all" of the outputs from deal(123), that number is not well defined.
Working these things out is not trivial -- but it is a really missing bit of the language.
There might be an opportunity for a syntax such as {}name -- e.g.,
arrayfun(@(X0) {}ode45(@f, tspan, X0), x0s)
meaning to gather all of the outputs of the call. At present, {} is not valid before a name.
Bruno,
Imagine that you want to implement
[temp{1:3}] = ndgrid(-1:.1:1);
C = cell2mat(cellfun(@(M) M(:), temp, 'uniform', 0);
That is, you want a single array in which each of the columns is one of the outputs of ndgrid(). And you want to do it as a single expression.
ndgrid() does not have a fixed number of outputs -- it is not like sin() with one fixed outputs, or max with two fixed outputs. nargout(@ndgrid) is -1 -- in other words the declaration is like
function varargout = ndgrid(varargin)
If you wanted to capture all of the outputs of max() then you could query ndgrid(@max) to get 2:
function outputs = gather_outputs(f, varargin)
n = nargout(f);
[outputs{1:n}] = f(varargin{:});
end
and you could
gather_outputs(@max, rand(3,5))
and this would be fine for gathering the two outputs of max into a single cell array.
But if we try to
gather_outputs(@deal, [])
then nargout(@deal) is -1 and if we said "Okay, take the absolute value of that -1" then we would be doing
[outputs{1:1}] = deal([])
which would give you just {[]} as the output.
This shows that you cannot just look at nargout() of the function you are invoking, such as @max or @deal .
Can we just look at nargout of the overall expression to determine the number of outputs to use for deal? No,
C = cell2mat(cellfun(@(M) M(:), gather_outputs(@ndgrid, -1:.1:1), 'uniform', 0));
would at best tell you that nargout is 1 (the C variable). You need something else to tell you the number of outputs you want to get -- something like
function outputs = gather_n_outputs(f, n, varargin)
[outputs{1:n}] = f(varargin{:});
end
and then you could
C = cell2mat(cellfun(@(M) M(:), gather_n_outputs(@ndgrid, 3, -1:.1:1), 'uniform', 0));
The request is to build this kind of facility in to MATLAB instead of having to write true functions like gather_n_outputs and have to pass function handles into them. Some syntax like
C = cell2mat(cellfun(@(M) M(:), {3}ndgrid(-1:.1:1), 'uniform',0));
where the hypothethical new syntax {3} indicates that you are requesting 3 outputs and that you want them gathered in a cell array. A common alternate syntax would be {} to request all of the outputs, like
C = cellfun(@(V) V.^2, {}max(rand(3,5)), 'uniform', 0)
which would hypothetically gather both (all) outputs of the max() call into a cell array that would then be available for further processing.
If you just used
C = cellfun(@(V) V.^2, max(rand(3,5)), 'uniform', 0)
then the knowledge of the single output would be carried through into the cellfun, which would tell max() to only emit a single output, so you would not get the second output processed. And the first output would be numeric not cell, so you would need arrayfun instead of cellfun...)
Why nargout doesn't meet what you ask?
function varargout = foo(varargin)
varargout = cell(1,nargout);
[varargout{:}] = deal('dummy');
fprintf('request with %d outputs\n', nargout);
Test
>> [A]=cellfun(@foo, {1 2 3}, 'UniformOutput', false);
request with 1 outputs
request with 1 outputs
request with 1 outputs
>> [A,B]=cellfun(@foo, {1 2 3}, 'UniformOutput', false);
request with 2 outputs
request with 2 outputs
request with 2 outputs
>>
A ready-built insertion method for tables, arrays, etc., so don't have to do all the under-the-hood indexing manually...given an array of the right size and matching index, if had table (say), then
tMyTable=insert(tMyTable,indices,tNewData);
ideally, an option with keyword 'Empty', T|F would be available as well.
Maybe I'm missing some magic pixie dust, but I've been unable to figure out a way to this without physically moving a section at a time to insert the new row which means working from rear to front to avoid changing the indices to the insertion points or by catenating the pieces starting from the front.
No, the feature/extension doesn't yet exist; the 'Before/After' position is one value, whether it is provided as index, variable name or logical vector. The logical vector, while a vector, can have only one true position.
I'm suggesting N new variables should be able to be added, each at some specified position -- use case/example is to add the column of means following each variable, for example. As is, they all have to go in one place and then get rearranged or added one-at-a-time.
I missed this item before. For table and timetable arrays this function has existed since release R2018a. See the addvars function.
Related to my previous re: insert() function for tables @ <What is Missing is insert() Function> would be enhancement to addvars to extend the indexing 'Before','After' index to be a vector of indices/existing variable names to all multiple variables to be added in one call at specified locations in the table.
As is, have to either add one-at-a-time where go or rearrange all afterwards. It may be a somewhat easier search to find the location want to go when adding by comparing to existing variable names before the new ones are added than after -- or simpler to put where want to begin with than construct the later indexing sequence.
Yeah, that's undoubtedly a better way on the construction, Walter -- I hadn't really thought about the implementation too much other than going "dead ahead" to "just get 'er done!" for the immediate task at hand.
Taking a break, I thought I'd throw it out as an idea for an enhancement for discussion...with the sidelight maybe somebody did have a "trick" hadn't thought of.
"what would the Empty option do?"
Insert blank record(s). With table you have to construct a record of the right size and type for each column and with the matching variable name; would just be "syntactic sugar" to have the routine automagically handle that as well. (Probably, the keyword is enough, likely the only use would be with the value 'T')
Useful if you know where the new record(s) goes(go) but don't yet have the data for it(them) at hand but will be delivered on the next train to arrive at the station. Or, if only have part of the full record with the rest not arriving until the container ship from China gets to port. Insert the blanks then fill in the fields later.
Possibly there's an 'Incomplete' option that would be the composite of the two steps for the partial data in hand case?
Not that big a deal for an array of native type, agreed, but have run into the case enough times with tables for the thought should be a builtin -- at least for the table/timetable/timeseries(?) classes.
You can do it without moving a section at a time by building a cumulative destination index that increases by the size of each gap at the appropriate places (though I need to think more about how to vectorize this step.) Then you assign the existing values into a new variable, destination indexed by the cumulative index. Then you can assign the new values into the gaps with a single assignment statement using a destination that is just the "holes".
Not sure exactly how to vectorize the index construction... but the data moving itself does not have to be done a section at a time.
Hmmm... what would the Empty option do?
Ability to convert Python items to MATLAB data types (other than numpy arrays) inside of MATLAB, rather than inside of Python using the admin-rights-required API: https://www.mathworks.com/help/matlab/matlab_external/install-the-matlab-engine-for-python.html?s_tid=srchtitle
There's an entry in the Release Notes for release R2022b that discusses converting scalar logical and numeric Python types to MATLAB types.
I'm not 100% sure if you had specific types in mind when you said "Python items" but as of release R2022a you can convert Python list and tuple types to MATLAB types.
Some (well, most) dynamic programming languages allow for an experienced developer to have an insight look into how their code is actually executed by the runtime environment.
They are able to see the parse tree and the compiled byte code.
They are able to see what parts of code are JIT-compiled, and how exactly the are compiled. When something is not compiled, they are able to see why.
The developer doesn't have to guess whether a particular optimization has kicked in or not. They know for sure how each and every object in their code is handled (whether it is CoW, passed by reference, passed by value).
I'd love to see these capabilities in MATLAB, too.
That doesn't make the function horrible, it just means the function is not suited to your needs. I don't complain that my hypothetical electric stove can't cool things down, even though with a TEC it is possible to both heat and cool down things with a single electrically powered device.
Although you could re-use a lot of the internals of readfile to fix double UTF-8 encoded files, the problem is not the function, it's the double encoding. Fixing double encoding is a different task from reading a file correctly. It wouldn't be a matter of 'mending my function', it would be re-using internal functions for a different goal. The word 'mending' implies my function is broken, but returning a double encoded file in a corrected form is not living up to the contract that the name of the function is offering.
Also, there are more reasons to close down the source than just thinking your users are dumb (suggesting otherwise sounds a lot like assuming bad faith). As an example: if you want to provide a function for a license fee, it would be plain stupid to have your license check in an m-file, since it would be almost trivial to circumvent.
> I tell people to use readfile(filename_or_url) and use that result, instead of telling them how to figure out if a file is UTF8 or ANSI and which exact conversion that requires for their specific OS and version of Matlab/Octave.
> ...
> The function is not suddenly horrible because you can't have a peek at the implementation and optimize your workflow accordingly. Why would the language as a whole be any different?
Given that I can't peek at the function's implementation, this function suddenly becomes horrible when I have a file double encoded to UTF-8 (and so your function returns garbage) and you considered your users too dumb to allow them to mend this function for their needs.
Same goes for the language.
The parse tree can be accessed through the undocumented mtree() function https://www.mathworks.com/matlabcentral/answers/180048-list-built-in-commands-used-by-m-function#answer_169083
(sorry for the wall of text)
I don't understand why people resort to calling the Matlab language bad (even if only by inference) if they disagree with the direction Mathworks has chosen. I don't think R is better as a language, or Python. They have their pros and cons. If I think a language has a con, why should I start calling it names? Do I have a right to say Python is stupid because you need to specify what exact version of what exact package you want to use, because otherwise everything will break or at least run the risk of not being able to reproduce the result for a paper? Of course not.
I don't think Matlab (as a language) is not mature. You may not like the direction that Mathworks is choosing to move the language, but that is something else.
It makes sense to hide implementation details on some level. Every time I post a function to the FEX I do exactly that. I tell people to use readfile(filename_or_url) and use that result, instead of telling them how to figure out if a file is UTF8 or ANSI and which exact conversion that requires for their specific OS and version of Matlab/Octave. If you want to know, read the m-file. I could have chosen to publish the function as a p-file, but I didn't. That was my call to make. The function is not suddenly horrible because you can't have a peek at the implementation and optimize your workflow accordingly. Why would the language as a whole be any different?
More on topic: I don't see why you would want to understand what the JIT is doing to your code, without wanting to optimize for that. What would be the use? I'm all for creating more understanding, but I don't see how this achieves more than what good coding practices would teach you.
I have no experience working with a parse tree, so I can't comment on that.
About that 'any professional programmer': that reeks of a true Schotsman fallacy. Apart from what I mentioned above, I don't see what more I could say. It sounds more like venting frustration than actually making a point.
@Bruno, it makes sense on some level to let speed be their business. It litterally is. GNU Octave has fewer debug tools, fewer implemented function, but most importantly for most people: it is a lot slower. If you write code following the guidelines everyone is being taught on this forum and in the documentation, Mathworks can focus engineering effort on only a single style of programming. conv used to be single-threaded, now I can saturate 12 threads with a convolution. The ismember function used to create memory issues for large amounts of text in a cell array (my use case: about 3-4 MB), now it works fast. The more you open up your internal mechanisms, the harder it is to make the fundamental changes required for orders of magnitude speedups.
None of this is to say that I am a big fan of every decision Mathworks makes. I just think there is enough valid criticism to level that you don't need to vent frustration under the guise of commentary. Just be honest with yourself and others.
When trying to understand these kinds of decisions, I find that it helps to keep in mind a couple of factors:
1) The purpose of the Free Software Foundation projects such as Octave is not to provide high quality or long term supported or "affordable even in developing countries" software. The purpose of the FSF is explicitly political: to make it effectively impossible for anyone anywhere in the world to "own" (control) what is commonly known as "intellectual property". The FSF is explicitly intended to destroy the economy of companies such as Microsoft, and Apple, and Mathworks that is based on "owning" the rights to sell software licenses. The FSF believes that once an intellectual idea is produced and instantiated, that it belongs to the entire world and should be available to everyone (**with** source code) for no more than a service fee for the time/resources to copy it. To the FSF, it is not necessary that companies such as Microsoft and Mathworks cease to exist, but rather that such companies should earn their income by providing consulting services to people who want modified versions of the software, or want programs written (with the new software being released in source code form to the public) -- or through sidelines in mechandising physical objects such as Mathworks-branded t-shirts, or Mathworks-branded toys for McDonalds Happy Meals. Therefore Mathworks is not just competing in a marketplace of providing high quality software that users are willing to pay for: Mathworks is competing in a marketplace where one of the significant players is determined to force Mathworks to give away the many millions of dollars invested into intellectual property development, for political reasons. The FSF actively wants proprietary software firms to fail in anything recognizable as their current forms, and the FSF has no plans to mitigate problems that would cause in the world: the FSF just has the nebulous idea that if software is worth doing, then someone will be willing to hire programmers to develop the software (that would then have to be given away to everyone.) Thus, to companies such as Mathworks, the FSF is not just a "respected competitor", it is a malignant actor determined to destroy the right of Mathworks to exist at all (except as a consulting firm.)
2) Google vs Oracle America https://en.wikipedia.org/wiki/Google_v._Oracle_America is an active legal case in the USA. To summarize very briefly: Sun developed Java and encouraged its wide adoption as a portable write-once platform; companies did that, including copying the major java interface components but writing their own version of the internals; Oracle bought Sun; Oracle now claims ownership of the very interface structure for Java and is trying to get companies to pay big $$$ for licensing what Sun once encouraged everyone to use. Oracle sued Google for $US8.8 billion in copyright damages with respect to Android. Google won in lower court, but the decisions were reversed in a higher court, and the matter is now on the docket of the US Supreme Court.
The relevance of this lawsuit to Mathworks is the question of whether Mathworks "owns" the structure and library interfaces it established for MATLAB (and so could potentially force FSF Octave project to modify or shut down). Likewise, if Mathworks were to expose the Execution Engine level, would Mathworks effectively be giving away those decades of intellecutal work for anyone to develop for free, or would they retain enough control to be able to prevent their competitors and organizations such as FSF from using the work against Mathworks? I know that if I were in the position of a company such as Mathworks, that I would not be willing to release significant information about internal infrastructure until the question of ownership / control of APIs was legally settled.
"If MathWorks wants to attract those professional programmers and not only position MATLAB as a "language for engineers and scientists", they will have to open up."
Unfortunately I think this won't happen. I have observed them to close down (and not open up) more and more over the years. Examples come to mind is: The mxArray internal structure was hidden, they removed access to undocumented functions, they advises user to code normally and should not care about understanding why some method is faster or slower, telling speed is their business.
Yair's undocument webpage has a huge success not without reason. And I can imagine it take a lot of effort to make it up-to-date.
What you request, and I'm 100% second, seems to hane negative correlation coefficient with TMW past/ current politics. They might have a reason to do it, mainly prevent plagiagism from competitors.
I'm not talking about optimizing for JIT. Having insight into what exactly JIT did, a developer can understand the reason why their code e.g. became slower in a newer release. Or why a particular construct is faster than another (without speculating with other MATLAB users as to why this is happening - which I seen here at FEX quite often).
You also seem to have missed my point about parse tree. This information can be usefull and can spawn a wide range of third-party MATLAB utilities - such as code analyzers, linters, profilers, dependency analyzers and so on.
But you are right that this is probably what MathWorks tries to avoid.
But having said that, there should be no doubt that any professional programmer will choose a decent programming language (such as Julia, R, Python or even Lua), which doesn't limit his (programmer's) knowledge and urge to dig as deep as he wants to dig.
If MathWorks wants to attract those professional programmers and not only position MATLAB as a "language for engineers and scientists", they will have to open up.
The problem with this idea is the long term portability of code. It is fairly easy to write code that will run on Matlab 6.5 (released in 2002) and R2020a, as you can see in my FEX submissions (although some of those functions are examples of when it is hard to so). If you want to optimize for the JIT engine of every release, you blow up your dev time. If your code runs 10% faster because you take more advantage of the JIT, but you spend weeks on that function, is it worth it?
The engineers at Mathworks are very aware of good coding practices taught to Matlab users, and tend to try to optimize for such code. There are absolutely more obscure parts of Matlab that allow speedups in unexpected places: cellfun(@isempty,data) versus cellfun('isempty',data) is an example of that.
As a last aside: diving deep into the inner workings of Matlab might be blocked by Mathworks for a completely different reason: if you can see how everything is put together it is possibly easier to reverse engineer, allowing competitors to leverage engineering time from Mathworks in their own products. I have no clue whether or not this is a consideration (since I don't have any insider knowledge, nor personally know anyone working at Mathworks), but it might be.
Thanks, Rik!
Thanks Rik!
I would love to have a command TeX2sym and sym2TeX. With it, we could bring formulas from LaTeX and run them in MATLAB and the other way around. It is a tedious work doing it by hand, and I could bet most MATLAB users do it or will do it eventually.
Another useful feature to add would be arbitrary-precision arithmetic...languages such as python and Java (with Bignum), allows unlimited precision when dealing with integers at least. Granted that It is slow, but I believe that the MATLAB team could do something better. And I honestly feel like MATLAB is missing a feature by not having it...even some competing software has it.
And finally I was considering buying the Text Analytics Toolbox, and it would be nice, if it could have a grammar/spell checker. Even if it is not as advanced or doesn't contain all words. With live scripts, we can write an interactive document, so it would be nice if MATLAB could correct our spelling (even if it required a ToolBox to do it).
Thank you for your interest, Walter and Rik. I believe that It is best to teach ourselves to code in a particular way, which will allow MATLAB to convert it to sym, than to be doing the task by hand every time.
Operator precedence could take care some of these issues. If \sin x + y was the input, then left-to-right precedence could dictate the output to be sin(x) + y. And to obtain sin(x+y), we would have to code in LaTeX \sin{x+y} or \sin(x+y).
Implied multiplication is indeed very common in mathematics and latex. On the other hand, it is also almost ubiquitous that variables are written just as single letters (especially if we are writing implied multiplication).
Finally, a command like tex2sym does not need to contemplate every math package imho, just some set of commands and macros. MATLAB could even return its best guess, and let us do the remainder of the task. Of course, ideally, we would just copy the contents from environments like $$ and \[ ... \] and paste them into MATLAB for a tex2sym conversion.
Wow , deep sir Walter!
For example
- sin x+y ->
%observed in the wild. People are likely to interpret as sin(x)+y . Not recommended latex style for trig functions - \sin x+y ->
%is it sin(x+y) or sin(x)+y? People are likely to interpret as sin(x)+y - \sin{x+y} -> %sin(x+y) but people are likely to guess sin(x)+y without being sure
- \sin{x}+y -> %sin(x)+y but people are likely to guess sin(x)+y without being sure
- sin(x+y) ->
%sin(x+y). Not recommended latex style for trig functions - \sin(x+y) ->
%sin(x+y). Better latex syntax for trig functions - \sin\left( x+y \right) ->
%sin(x+y) . Considered better latex style because it allows latex to match heights of () when internal content is varying sizes, but has too much space - mx+b ->
%m*x+b which people understand from long convention. But it could be a variable named mx that is being added to b . This is, however, the primary recommended latex style - m.x+b ->
%m*x+b but not recommended latex style. However, lower dot for multiplication tends to show up in papers that have matrix multiplication - m*x+b ->
%m*x+b but actively recommended against latex style - {m}{x}+b ->
%m*x+b which people understand from long convention, but visually it could be a variable named mx that is being added to b. Easier for mechanical interpretation but not typical latex style - m{\cdot}x+b ->
%m*x+b understandable both mechanically and human. One of the recommended latex styles - m{\times}x+b ->
-> %m*x+b understandable both mechanically and human, but less common in practice. Considered to be one of the valid latex styles if needed - f{\textrm{\"{o}}}rl{\textrm{\aa}}t ->
%a single word. Using {} inside an expression does not reliably indicate multiplication. And it was harder to get the special characters to work here than it should have been
Implied multiplication without any syntactic separation is very common in mathematics and latex.
The above shows just some of the ways that simple operations can be written in Latex, and tex2sym would have to try to understand them all.
There is also the issue that a lot of latex uses constructs that MATLAB does not support, especially \usepackage and amsmath mode.
sym2tex exists as latex()
tex2sym is pretty hard. Pretty mathematical notation is often ambiguous.
Neat idea. With enough patience it shouldn't be too hard to write a basic tex2sym, but I don't have any good ideas for sym2tex that wouldn't require a lot of cleaning up.
I think the main problem with tex2sym would be interpreting the subscript and superscript. The main problem with sym2tex would probably be that a clear and concise mathematical description is not necessarily the same as compact Matlab code, even when working with symbolic variables.
facet_wrap or facet_grid (or general ggplot2 functionality) version of subplots, or some altered, simpler customizability for subplotting subsets of data.
gramm is an available toolbox, but it'd be nice to have it built in
See Also
Tags
You can also select a web site from the following list
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)
Asia Pacific
- Australia (English)
- India (English)
- New Zealand (English)
- 中国
- 日本Japanese (日本語)
- 한국Korean (한국어)