functionend ignored

[David Grundberg]

John W. Eaton skrev:
> On 13-Jul-2009, David Grundberg wrote:
>
> | Hi,
> | 
> | been working on lint, but I got stuck on this behavior I don't agree with.
> | 
> | Bug report for Octave 3.1.55 configured for i686-pc-linux-gnu
> | 
> | hg tip:
> | changeset:   9431:bfc7b000a229
> | tag:         tip
> | user:        John W. Eaton <jwe at octave.org>
> | date:        Sat Jul 11 12:46:10 2009 -0400
> | summary:     file-ops.cc (file_ops::symlink, file_ops::readlink): avoid 
> | incorrectly sized buffer
> | 
> | Description:
> | -----------
> | 
> |   * A user function may end with any number of 'functionend's, and no
> |     error will be given.
> | 
> | Repeat-By:
> | ---------
> | 
> |   1. Create a file buggy.m:
> | 
> | cat > buggy.m << EOF
> | function [] = buggy()
> | endfunction
> | endfunction
> | EOF
> | 
> |   2. Run octave like
> | 
> | /opt/octave-head/bin/octave --eval 'addpath("."); __display_tokens__(1); 
> | buggy'
> | 
> |   3. Expected behavior is a syntax error. But the experienced behavior
> |      is that the function runs as normal.
> | 
> |        Note that running
> | 
> | /opt/octave-head/bin/octave  --eval '__display_tokens__(1);' buggy.m
> | 
> |      does raise a syntax error because of the extra endfunction.
> | 
> | Comments:
> | --------
> | 
> |   * Ends seems to be ignored? In the following excerpts, note that the
> |     end token is replaced with a '\n' in the first run.
> | 
> |     Running as a function from the Octave prompt:
> | 
> | octave:1> buggy
> | NAME
> | \n
> | FCN
> | [
> | ]
> | '='
> | NAME
> | (
> | )
> | \n
> | \n
> | \n
> | END_OF_INPUT
> | 
> | parsed function end as end of input!
> | END_OF_INPUT
> | 
> | octave:2>
> | 
> |     Running buggy.m as an argument to octave:
> | 
> | FCN
> | [
> | ]
> | '='
> | NAME
> | (
> | )
> | \n
> | END
> | parsed function end as a end token!
> | \n
> | END
> | parse error near line 3 of file buggy.m
> | 
> |   syntax error
> | 
> |  >>> endfunction
> | 
> |     For completeness, here are the changes I have done to the tip:
>
> OK, what is happening is that endfunction tokens (and end tokens that
> should match function keywords) are ignored by the lexer.  I think
> this was done as a semi-tricky way to handle subfunctions.  I guess we
> should fix that in some other way so that the parser does see end
> keywords instead of just EOF.  I'll try to take a look a this part of
> the problem, but I can't guarantee that I will get to it any time
> soon.
>
> jwe
>   
Looked further into the lexer yesterday and pretty much came to the same 
conclusion (special cases for parsing subfunctions). I think this is an 
interesting problem (and such a challenging tokenizer/parser) so I'll 
look at it more.

regards,
David