DFS Links and Timeouts

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

DFS Links and Timeouts

Martin D. Pedersen
Hi jcifs guys

I'm having some problems with DFS Links.

I have the following DFS setup.
   //primary/dfs/link   link point to //secondary/data

When I traverse //primary/dfs/link I am transparently redirected to
//secondary/data (in theory).

But jcifs somehow timeouts when accessing files through dfs links.
If i traverse the files directly from //secondary/data there is no
problem!

Here is my example code:

import jcifs.smb.SmbFile;

public class SmbTimeout {
       
        public static void jcifsScan(String root, String userName,
String password, int sleepTime) throws Exception {
            System.setProperty("jcifs.smb.client.username",userName);
            System.setProperty("jcifs.smb.client.password",password);
            long start = System.currentTimeMillis();
            SmbFile smbRoot = new SmbFile(root);
            SmbFile[] files = smbRoot.listFiles();
            for(SmbFile f : files) {
            System.out.println(f.canRead()+" :
"+(System.currentTimeMillis()-start));
                Thread.sleep(sleepTime);
            }
           
        }
               
        public static void main(String[] p_args) throws Exception {
                if(p_args.length!=4) {
                        System.out.println("Usage: <smbroot> <username>
<password> <sleeptime(ms)>");
                        return;
                }
                String smbRoot = p_args[0];
                String userName = p_args[1];
                String password = p_args[2];
                int sleepTime = Integer.parseInt(p_args[3]);
                jcifsScan(smbRoot,userName,password,sleepTime);
        }
               
}

When running:
Java -classpath .:jcifs-1.2.7.jar SmbTimeout smb://primary/dfs/link/
user password 16000
I get:
true : 744
false : 16760
false : 32777
[...]

But when I run:
Java -classpath .:jcifs-1.2.7.jar SmbTimeout smb://secondary/data/ user
password 16000
I get:
true : 560
true : 16597
true : 32632
[...]

The reason this is a problem is because I'm implementing a crawler kind
of process, that needs to process the files. Sometimes for more than
16s.
I realise that I can set jcifs.smb.client.soTimeout=0 and it will solve
my immediate problem.
But I fear that this is a generel problem in handeling DFS referals.

-- Martin D. Pedersen
Reply | Threaded
Open this post in threaded view
|

Re: DFS Links and Timeouts

Michael B Allen-4
On Fri, 16 Dec 2005 15:38:30 +0100
"Martin D. Pedersen" <[hidden email]> wrote:

> But jcifs somehow timeouts when accessing files through dfs links.
> If i traverse the files directly from //secondary/data there is no
> problem!

I *think* I have this fixed. Try the attached patch and let us know how
it goes for you.

Basically the problem is that the member SmbFile.unc (which
is a path relative to the share) is being changed when a
DFS referral occurs. Initially this isn't a problem but after
jcifs.smb.client.attrExpirationPeriod expires SmbFile.unc is used to
construct paths that can be incorrect.

> The reason this is a problem is because I'm implementing a crawler kind
> of process, that needs to process the files. Sometimes for more than
> 16s.
> I realise that I can set jcifs.smb.client.soTimeout=0 and it will solve
> my immediate problem.

I don't think that would help actually. Actually it would just keep
transports open indefinitely and you would quickly exhaust all memory.

For a crawler you probably want to increase attrExpirationPeriod greatly
though. That in combination with using a tree traversal algorithm that
does not built large lists of nodes that will later become stale will
give you much better results.

Mike

DfsTimeout.patch (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: DFS Links and Timeouts

Martin D. Pedersen
In reply to this post by Martin D. Pedersen
 

> -----Original Message-----
> From: Michael B Allen [mailto:[hidden email]]
> Sent: 17. december 2005 22:47
> To: Martin D. Pedersen
> Cc: [hidden email]
> Subject: Re: [jcifs] DFS Links and Timeouts
>
> On Fri, 16 Dec 2005 15:38:30 +0100
> "Martin D. Pedersen" <[hidden email]> wrote:
>
> > But jcifs somehow timeouts when accessing files through dfs links.
> > If i traverse the files directly from //secondary/data there is no
> > problem!
>
> I *think* I have this fixed. Try the attached patch and let
> us know how it goes for you.

The patch fixes the (originally) problem, but there is still another
problem.

In my first test code I only used SmbFile.canRead(), this now works
fine.
If I extend the test code to also call SmbFile.length() it fails again,
but this time with an Exception.

Exception in thread "main" jcifs.smb.SmbException: The system cannot
find the path specified.
        at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:503)
        at jcifs.smb.SmbTransport.send(SmbTransport.java:580)
        at jcifs.smb.SmbSession.send(SmbSession.java:229)
        at jcifs.smb.SmbTree.send(SmbTree.java:102)
        at jcifs.smb.SmbFile.send(SmbFile.java:688)
        at jcifs.smb.SmbFile.queryPath(SmbFile.java:1196)
        at jcifs.smb.SmbFile.length(SmbFile.java:2146)
        at
visanti.crawler.samba.SmbTimeout.jcifsScan(SmbTimeout.java:19)
        at visanti.crawler.samba.SmbTimeout.main(SmbTimeout.java:35)

If I only call SmbFile.length() this Exception doesn't occur!
I guess there are more path rebuilds to be found?

-- Martin

Reply | Threaded
Open this post in threaded view
|

Re: DFS Links and Timeouts

Michael B Allen-4
On Mon, 19 Dec 2005 12:38:03 +0100
"Martin D. Pedersen" <[hidden email]> wrote:

> > > But jcifs somehow timeouts when accessing files through dfs links.
> > > If i traverse the files directly from //secondary/data there is no
> > > problem!
> >
> > I *think* I have this fixed. Try the attached patch and let
> > us know how it goes for you.
>
> The patch fixes the (originally) problem, but there is still another
> problem.
>
> Exception in thread "main" jcifs.smb.SmbException: The system cannot
> find the path specified.
After looking closely at how paths are handled across DFS I found a few
problems. I think I understand this well now but I can't be certain all
cases will work properly. Please test the attached patch carefully. If
you find a problem please let me know and we'll rinse and repeat.

Thanks,
Mike

DfsPathFix.patch (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: DFS Links and Timeouts

Michael B Allen-4
Patch applied. To be released RSN.

On Mon, 19 Dec 2005 16:47:25 -0500
Michael B Allen <[hidden email]> wrote:

> On Mon, 19 Dec 2005 12:38:03 +0100
> "Martin D. Pedersen" <[hidden email]> wrote:
> > > > But jcifs somehow timeouts when accessing files through dfs links.
> > > > If i traverse the files directly from //secondary/data there is no
> > > > problem!
> > >
> > > I *think* I have this fixed. Try the attached patch and let
> > > us know how it goes for you.
> >
> > The patch fixes the (originally) problem, but there is still another
> > problem.
> >
> > Exception in thread "main" jcifs.smb.SmbException: The system cannot
> > find the path specified.
>
> After looking closely at how paths are handled across DFS I found a few
> problems. I think I understand this well now but I can't be certain all
> cases will work properly. Please test the attached patch carefully. If
> you find a problem please let me know and we'll rinse and repeat.
>
> Thanks,
> Mike
>