Virtual Directories, Virtual Files

Comments:

Author Thread:

Jairo.Lopez

Virtual Directories, Virtual Files
Posted: Tuesday, May 04, 2004 4:38 PM (EST)

Since there are several parts to your question I'm not certain I know exactly what you are asking.
My understanding of the first part is that you would like to have wildcards in the virtual path, something like:
/images/*

When the server gets a request for /images/aplglyph.gif, the request will result in a call to the workspace.
This is a great idea, however, to keep the performance and define a consistent interface we need to address how the server will resolve virtualpaths, such as:
/images/*
/images/banners/*
/banners/*

How would the following be resolved:
/apl/images/mypic.gif

Should the server return a 404 or call the workspace?
The second part of your question is about handling the 404 error. We could add the ability to specify a virtualpath as a custom error.
We need to be very conscious that adding features has the potential to impact server performance. Since we are now significantly faster than IIS we want to make certain we enhance the server while maintaining this advantage.



brent.hildebrand

Virtual Directories, Virtual Files
Posted: Tuesday, May 04, 2004 7:36 PM (EST)

Perhaps Fred Waid will remember the discussion I had with him at the 2003 Users Conference where he felt that this could be implemented with minimal impact. I think the option was that if the a default handler was not needed and thus was not enabled, it would require minimal overhead.

WebServices would handle the following fine:

http://webaddress/SpreadSheet?Month=06&Year=2004&User=Hildebrand

But, the following could not be implemented for arbitrary months and years.

http://webaddress/SpreadSheet/Hildebrand062004.xls

After the 2003 conference, I wrote a simple web server in APL, using []NI to serve out files and pictures to my close family and friends. Read, not mission critical. There are hundreds of virtual links to files that are only generated when requested. Currently it is running here:

http://68.65.28.160:20

It could be written to run under WebServices. But each file would need to be generated and written to disk. Fred???



fred.waid

Virtual Directories, Virtual Files
Posted: Wednesday, May 05, 2004 10:20 AM (EST)

We have been looking at adding wild cards to the virtual paths for some time.

The way that seems to make the most sense is to use the startswith method, so /image/* would refer all references to virtual paths that begin with /image/ to the associated workspace.

Conversely /image would not be referred, nor would /imagemine, etc.

If a request is made for a resource that does not exist, or for that matter any request which would generate an error, the custom error could be set to a virtual path, which would cause the request to be referred to the appropriate workspace.

When returning a gif or for that matter any binary or application type file, you should use the document-filename as the return option to avoid encoding issues. You can set the document-filename-delete to indicate if the file is to be deleted after it is returned to the requesting process. This has the added benefit of reducing memory usage, and makes returns to requests from slow connections less onerous on the server.

By the way your sunset pictures from last conference are really amazing!



j.merrill

Virtual Directories, Virtual Files
Posted: Wednesday, May 05, 2004 10:36 AM (EST)

(Jairo, can you tell me how I can format my text so that it's in separate paragraphs, or can use different fonts, like you did in your message? I've tried using <br> and putting <html></html> around my text, but it always previews the same way. It's supposed to be 5 separate paragraphs!) It seems from your answer that the wildcard feature has not yet been added. Is that right? Having a feature where the workspace has an option to be called for 404 errors (and possibly other ones) would not add any processing time, except for requests that would have returned 404. Wouldn't it be unusual to have a benchmark test that checks how fast the server responds to requests for pages that don't exist? Having the server be fast when it doesn't talk to APL at all is not a case where we want to brag that it's faster than IIS! Here's a situation where you would do well to be using an "open source" model for at least parts of the code. As you say, defining a consistent interface for resolving virtual paths is important. In the server code, there is presumably an array that represents the different virtual directories (as objects of some type, or possibly as an array of strings that are the virtual directory paths); there is logic that takes the relevant part of the URL and tries to find a match. That logic could be placed in a separate routine that we could enhance -- or a mechanism could be built so that if I want to provide my own assembly with a custom implementation, it gets called. The result of keeping the code private is likely to be that someone will decide that they could build an equivalent tool -- it might not work as well, and might not be as complete, but at least the programmer who did it would have complete control. APLers are well known for re-implementing functionality that others have built. We should be able to do a better job when the code is in a language (C#) that's more supportive of separate developers working on the same project (due to interfaces, multiple source files, and larger programs in general).


fred.waid

Virtual Directories, Virtual Files
Posted: Wednesday, May 05, 2004 12:34 PM (EST)

We are just finishing a new source code control site which will be integrated with the new APLDN and is designed to allow developers to post projects to the APL servers and share code development.   All of the code is encrypted before it is posted to the servers, so only people you designate will be able to access the project or see the code. It will never exist on our servers except in the encrypted format.

Initially it is designed specifically for APL projects, and we have a couple of interesting ones which we are going to put up.

The first will be our APL SQL Database project which actually is as fast or faster than MS SQL Server for record access. We also have a version of this in C#, which means that databases you create with APL can also be accessed by .Net projects.



j.merrill

Virtual Directories, Virtual Files
Posted: Wednesday, May 05, 2004 2:26 PM (EST)

That's good to know, but I'm hoping that you might share the code for the web server (even though it's in C# so some won't want to look). It would be nice to be able to enhance it, possibly speed it up (though that seems unimportant at this point), make sure that the same things that are done in the GUI can be done under program control, and so on. Asking again, can I learn how to at least make new paragraphs in my replies? I had hoped this would be a new paragraph but it's not...


Jon.Capps

Virtual Directories, Virtual Files
Posted: Wednesday, May 05, 2004 5:33 PM (EST)

When I post a message, I have a tool bar at the top of the input window.

It lets me bold, Italic, and Underline

You can also paste code!

Turn on and off the APL Font

Etc. Color and Hi lite and some other tricks. Is this what you were asking?

Jon.



William.Rutiser

Tool bar( was Virtual Directories, Virtual Files)
Posted: Wednesday, May 05, 2004 7:01 PM (EST)

The toolbar seems to require Internet Explore. I see it with IE5.5 but not with Opera 6.03 or Opera 7.23


brent.hildebrand

Tool bar( was Virtual Directories, Virtual Files)
Posted: Thursday, May 06, 2004 12:55 AM (EST)

I some times see the toolbar in IE6, but most of the time I don't. Also, as I type this, the edit window starts as a single line 3/5ths of the way down the screen. There appears to be a band where the toolbar should be, but it is not displaying. I assume this is a Java thing - write once, debug everywhere.


davin.church

Tool bar( was Virtual Directories, Virtual Files)
Posted: Thursday, May 06, 2004 1:21 AM (EST)

I've seen that happens once or twice before, but pressing Refresh fixes the problem so I haven't been able to replicate it.



j.merrill

Virtual Directories, Virtual Files
Posted: Thursday, May 06, 2004 11:25 AM (EST)

I've switched computers, to one with IE 6 -- and now I see the toolbar. I didn't with IE 5.0. Thanks for the answer.

The cursor is shaking in a ridiculous fashion, and the screen almost falls behind my typing (though I'm fast, that's ridiculous for a 1ghz machine).

I guess it's time to update IE on my main machine -- I'm just worried that "nothing will ever be the same" after I do so...



fred.waid

Virtual Directories, Virtual Files
Posted: Thursday, May 06, 2004 11:48 AM (EST)

One of the things we want to be able to do with these forums is to provide a fast track to product development.

Of the features we have discussed here the wild card is one that everyone needs.

So, /image/* will point all paths that start with /image/ to the associated workspace and function.

As for mapping real paths to virtual paths directly we are somewhat concerned about security issues. At this point, only the public folder specified in the server setup will ever be accessed by the APL+WebServices to return real files.   If we support mapping real paths to virtual paths, we can end up with unintended consequences. IIS does this, and it is a continual security issue for them that they have to patch.

It seems that it would be preferable to use the wildcard approach and have an APL function return the filename and path. That way a developer has made an explicit decision in a function as to what to expose and the security is an application issue and not a server issue.

The interface between server and APL session is so fast that there is very little overhead in having APL select the file to return as opposed to having the server make the decision. Depending on how the APL funciton is written, it could even work out to be faster.

We will be looking at making a beta version of the APL+WebServices available with this included.



davin.church

Virtual Directories, Virtual Files
Posted: Thursday, May 06, 2004 12:08 PM (EST)

Mapping virtual to real paths could be restricted to the web site's home directory tree and still provide most of the benefits without the security risks.


brent.hildebrand

Virtual Directories, Virtual Files
Posted: Thursday, May 06, 2004 12:09 PM (EST)

Wildcard, "*" in Virtual Paths will be good. This gives APL the power to have control of what is returned by the web request. Linking an error, such as 404, to a workspace and function would provide similar functionality, with minimal overhead as it would only be triggered when a file does not exsist, but again gives the APL application ultimate control. Both would provide for virtual files.

And again - no toolbar in IE6 in the message editor.



fred.waid

Virtual Directories, Virtual Files
Posted: Thursday, May 06, 2004 12:19 PM (EST)

Yes, we are also including the ability to set a virtual path as a custom error. Being able to create the results for 404 errors should give you lots of interesting options.

We also have just completed a com interface which exposes as methods and properties the functionality of the webservice interface. We will also put this in as beta code. Then everyone can really get in trouble reconfiguring the server under program control.



brent.hildebrand

Virtual Directories, Virtual Files
Posted: Thursday, May 06, 2004 12:35 PM (EST)

OK - I should wait until the beta, but I just got to ask, will APL be used to start WebServices, or will WebServices be able to pass a pointer to WebServices object to APL so APL can access the methods and properties? This IS sounding fun!


davin.church

Virtual Directories, Virtual Files
Posted: Thursday, May 06, 2004 12:42 PM (EST)

Letting APL code deal with (missing) virtual paths is sufficient, but it would really be nice to have a built-in (configure-time) method for handling it so you don't have to write code to do the simple stuff.


fred.waid

Virtual Directories, Virtual Files
Posted: Thursday, May 06, 2004 12:47 PM (EST)

You are one step ahead :-)

You have the webserver, which is one application running as a service.

You have the UI which is a separate application which then talks to the service interface and then the service interface talks to the webserver. We can not actually talk directly to the service without violating security issues. Which we did not want to do.

So, the new com provides the same capabilities of the UI but as a com with methods and properties.

But you are right, we are working on a com version of the server so that APL can act as a server manager for truly wild applications. So you can multiple servers running multiple webservers all controlled by APL.

So, what are we going to do tomorrow? Try to take over the world!



brent.hildebrand

Virtual Directories, Virtual Files
Posted: Saturday, May 08, 2004 5:04 AM (EST)

I get 404 when I try to download the new WebServer... :(


fred.waid

Virtual Directories, Virtual Files
Posted: Saturday, May 08, 2004 10:04 AM (EST)

Try the download again now, it should work fine. If you see the 404 error, just hit refresh on the 404 page, and that will clear the browser cache. Then go back and click download.   We have installed a new server and ATT circuit to speed up the download and uploads.