[Report]Automated testing of Samba with Windows Protocol Test Suites

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

[Report]Automated testing of Samba with Windows Protocol Test Suites

Samba - samba-technical mailing list
Hi everyone,

Happy new year!

I've done some job to use Ansible and OpenStack Cloud to automate the
testing of Samba with Windows Protocol Test Suites
<https://github.com/Microsoft/WindowsProtocolTestSuites>.

So far, I've only tried KDC category, but the other ones are easy to
support.

This is the current KDC test result:

total: 85

inconclusive: 3

passed: 32

failed: 50

And this is the 50 failed tests:

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CanonicalizeSpnInReferralTgt
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.UserInAnotherDomain
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.ComputerAccount_DES_PreAuthentication_Fail
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.DesDowngradeProtection
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.HwAuthentFlag
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.Krbtgt_DES_PreAuthentication_Fail
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.OkAsDelegateFlag
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccount_DES_PreAuthentication_Fail
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountDisabled
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountExpired
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountLocked
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountPasswordMustChangePast
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountPasswordMustChangeZero
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserDelegationNotAllowed
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserPreAuthenticationNotRequired
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UsingPasswordWithDES
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadMatch
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadOptions
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadProtocolVersionNum
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorCannotPostdate
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorFieldTooLong
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorGeneric
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorMsgType
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorRepeat
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTicketNeverValid
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTktExpired
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTktNotYetValid
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.AdFxFastArmorInAuthenticator
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.AdFxFastUsedInAuthenticator
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.FastKrb_Error
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.FastOptions_HideClientNames
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.KrbFastFinishedAsRep
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.KrbFastFinishedTgsRep
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.PaFxFastAdvertise
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.StrengthenKey
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.UnsupportedFastArmorType
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileSiloTest.Protected_Users_Interactive_Logon_Computer_A2A2_Fail
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileSiloTest.Protected_Users_Interactive_Logon_Computer_A2A2_Succeed
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.InteractiveLogonUseExplicitFast
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.InteractiveLogonUseFast
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.APDS_KERBEROS_PAC_VALIDATION
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.ServerSignature
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_AdFxFastUsedInAuthenticator
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_FastKrb_Error
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_InteractiveLogonUseExplicitFast
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_InteractiveLogonUseFast
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_KrbFastFinishedAsRep
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_KrbFastFinishedTgsRep
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_StrengthenKey
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_UnsupportedFastArmorType


It's now quite easy to set up the testing network and run it, and we are
going to fix the failed ones gradually in future.

--
Joe Guo
[hidden email]
Catalyst IT

Reply | Threaded
Open this post in threaded view
|

Re: [Report]Automated testing of Samba with Windows Protocol Test Suites

Samba - samba-technical mailing list
On Wed, Jan 03, 2018 at 10:05:43AM +1300, joeg--- via samba-technical wrote:

> Hi everyone,
>
> Happy new year!
>
> I've done some job to use Ansible and OpenStack Cloud to automate the
> testing of Samba with Windows Protocol Test Suites
> <https://github.com/Microsoft/WindowsProtocolTestSuites>.
>
> So far, I've only tried KDC category, but the other ones are easy to
> support.
>
> This is the current KDC test result:
>
> total: 85
>
> inconclusive: 3
>
> passed: 32
>
> failed: 50
>
> And this is the 50 failed tests:
>
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CanonicalizeSpnInReferralTgt
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.UserInAnotherDomain
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.ComputerAccount_DES_PreAuthentication_Fail
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.DesDowngradeProtection
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.HwAuthentFlag
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.Krbtgt_DES_PreAuthentication_Fail
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.OkAsDelegateFlag
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccount_DES_PreAuthentication_Fail
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountDisabled
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountExpired
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountLocked
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountPasswordMustChangePast
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountPasswordMustChangeZero
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserDelegationNotAllowed
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserPreAuthenticationNotRequired
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UsingPasswordWithDES
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadMatch
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadOptions
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadProtocolVersionNum
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorCannotPostdate
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorFieldTooLong
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorGeneric
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorMsgType
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorRepeat
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTicketNeverValid
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTktExpired
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTktNotYetValid
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.AdFxFastArmorInAuthenticator
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.AdFxFastUsedInAuthenticator
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.FastKrb_Error
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.FastOptions_HideClientNames
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.KrbFastFinishedAsRep
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.KrbFastFinishedTgsRep
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.PaFxFastAdvertise
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.StrengthenKey
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.UnsupportedFastArmorType
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileSiloTest.Protected_Users_Interactive_Logon_Computer_A2A2_Fail
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileSiloTest.Protected_Users_Interactive_Logon_Computer_A2A2_Succeed
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.InteractiveLogonUseExplicitFast
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.InteractiveLogonUseFast
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.APDS_KERBEROS_PAC_VALIDATION
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.ServerSignature
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_AdFxFastUsedInAuthenticator
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_FastKrb_Error
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_InteractiveLogonUseExplicitFast
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_InteractiveLogonUseFast
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_KrbFastFinishedAsRep
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_KrbFastFinishedTgsRep
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_StrengthenKey
> Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_UnsupportedFastArmorType
>
>
> It's now quite easy to set up the testing network and run it, and we are
> going to fix the failed ones gradually in future.

Great work Joe - thanks for the report !

Jeremy.

Reply | Threaded
Open this post in threaded view
|

Re: [Report]Automated testing of Samba with Windows Protocol Test Suites

Samba - samba-technical mailing list
In reply to this post by Samba - samba-technical mailing list
On Tuesday, 2 January 2018 22:05:43 CET joeg--- via samba-technical wrote:
> Hi everyone,

Hi Joe,

> Happy new year!

same to you!

> I've done some job to use Ansible and OpenStack Cloud to automate the
> testing of Samba with Windows Protocol Test Suites
> <https://github.com/Microsoft/WindowsProtocolTestSuites>.
>
> So far, I've only tried KDC category, but the other ones are easy to
> support.
>
> This is the current KDC test result:
>
> total: 85
>
> inconclusive: 3
>
> passed: 32
>
> failed: 50

Could please you also run this with MIT Kerberos?


        Andreas

--
Andreas Schneider                   GPG-ID: CC014E3D
Samba Team                             [hidden email]
www.samba.org



Reply | Threaded
Open this post in threaded view
|

Re: [Report]Automated testing of Samba with Windows Protocol Test Suites

Samba - samba-technical mailing list
Hi Andreas:

This is the result for Kerberos test suite:

total: 168

inconclusive: 5

passed: 37

failed: 126

And these are the failed ones:

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.AZOD.AZODSyntheticTest.CBAC_Smb2_Negative_AccessFile
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.AZOD.AZODSyntheticTest.CBAC_Smb2_Possitive_AccessFile
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.AZOD.AZODSyntheticTest.CrossRealm_Smb2_Negative_AccessFile
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.AZOD.AZODSyntheticTest.CrossRealm_Smb2_Possitive_AccessFile
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.AZOD.AZODSyntheticTest.DAC_Smb2_Negative_AccessFile
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.AZOD.AZODSyntheticTest.DAC_Smb2_Possitive_AccessFile
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Claim.ClaimsTest.Kerberos_CrossRealm_ADSource_User_Only_Transform_From_Config
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Claim.ClaimsTest.Kerberos_CrossRealm_ADSource_User_Only_Transform_Use_AD_and_CTA
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Claim.ClaimsTest.Kerberos_SingleRealm_ADSource_User_Only
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CanonicalizeSpnInReferralTgt
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CrossRealm_KdcSignature
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CrossRealm_KERB_VALIDATION_INFO
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CrossRealm_OtherOrgSIDinPACFailure
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CrossRealm_OtherOrgSIDinPACSuccess
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CrossRealm_PAC_CLIENT_INFO
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CrossRealm_PAC_DEVICE_CLAIMS_INFO
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CrossRealm_PAC_DEVICE_INFO
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CrossRealm_ReferralTgs_NoOkAsDelegateFlag
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CrossRealm_ServerSignature
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CrossRealm_UPN_DNS_INFO
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.ForwardableTicket_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.ForwardableTicket_Ldap
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.ForwardableTicket_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.Pa_Svr_Referral_Info
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.ReferralTicketIncludePacByDefault_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.ReferralTicketIncludePacByDefault_Ldap
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.ReferralTicketIncludePacByDefault_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.ReferralTicketWithPac_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.ReferralTicketWithPac_Ldap
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.ReferralTicketWithPac_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.ReferralTicketWithPacAndClaim_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.ReferralTicketWithPacAndClaim_Ldap
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.ReferralTicketWithPacAndClaim_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.TransitedPolicyCheckedFlag
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.UserInAnotherDomain
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileHttpApTest.ChannelBindingSuccess_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileHttpApTest.DetectAuthenticatorModification_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileHttpApTest.DetectTicketModification_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileHttpApTest.RequestDeviceClaim_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileHttpApTest.RequestUserClaim_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileHttpApTest.RestrictedKrbHost_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileHttpApTest.ServicePrincipalName_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileHttpApTest.ServiceTicketWithoutPac_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileHttpApTest.TokenRestrictionDifferentMachineId_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileHttpApTest.TokenRestrictionSameMachineId_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileHttpApTest.UsingFAST_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.ComputerAccount_DES_PreAuthentication_Fail
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.DesDowngradeProtection
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.HwAuthentFlag
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.Krbtgt_DES_PreAuthentication_Fail
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.OkAsDelegateFlag
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccount_DES_PreAuthentication_Fail
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountDisabled
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountExpired
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountLocked
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountPasswordMustChangePast
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountPasswordMustChangeZero
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserDelegationNotAllowed
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserPreAuthenticationNotRequired
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UsingPasswordWithDES
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadMatch
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadOptions
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadProtocolVersionNum
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorCannotPostdate
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorFieldTooLong
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorGeneric
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorMsgType
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorRepeat
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTicketNeverValid
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTktExpired
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTktNotYetValid
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileSmb2ApTest.ChannelBindingSuccess_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileSmb2ApTest.DetectAuthenticatorModification_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileSmb2ApTest.DetectTicketModification_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileSmb2ApTest.RequestDeviceClaim_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileSmb2ApTest.RequestUserClaim_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileSmb2ApTest.RestrictedKrbHost_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileSmb2ApTest.ServicePrincipalName_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileSmb2ApTest.ServiceTicketWithoutPac_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileSmb2ApTest.TokenRestrictionDifferentMachineId_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileSmb2ApTest.TokenRestrictionSameMachineId_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileSmb2ApTest.UseDESOnly_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileSmb2ApTest.UsingFAST_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.AdFxFastArmorInAuthenticator
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.AdFxFastUsedInAuthenticator
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.FastKrb_Error
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.FastOptions_HideClientNames
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.KrbFastFinishedAsRep
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.KrbFastFinishedTgsRep
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.PaFxFastAdvertise
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.StrengthenKey
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.UnsupportedFastArmorType
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileSiloTest.Protected_Users_Interactive_Logon_Computer_A2A2_Fail
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileSiloTest.Protected_Users_Interactive_Logon_Computer_A2A2_Succeed
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileSiloTest.Protected_Users_Network_Logon_Computer_A2A2_Succeed
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.CrossRealmGetReferralTGT
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.CrossRealmNetworkLogonSmb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.InteractiveLogonUseExplicitFast
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.InteractiveLogonUseFast
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.NetworkLogonClaimsSmb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.NetworkLogonCompoundIdentitySmb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.NetworkLogonHttp
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.NetworkLogonSmb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.APDS_KERBEROS_PAC_VALIDATION
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.PAC_CLIENT_CLAIMS_INFO
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.PAC_DEVICE_CLAIMS_INFO
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.PAC_DEVICE_INFO
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.ServerSignature
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_AdFxFastUsedInAuthenticator
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_CrossRealm_PAC_DEVICE_CLAIMS_INFO
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_CrossRealm_PAC_DEVICE_INFO
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_FastKrb_Error
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_InteractiveLogonUseExplicitFast
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_InteractiveLogonUseFast
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_KrbFastFinishedAsRep
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_KrbFastFinishedTgsRep
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_NetworkLogonCompoundIdentitySmb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_PAC_CLIENT_CLAIMS_INFO
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_PAC_DEVICE_CLAIMS_INFO
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_PAC_DEVICE_INFO
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_RequestDeviceClaim_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_RequestDeviceClaim_Smb2
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_StrengthenKey
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_UnsupportedFastArmorType
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_UsingFAST_Http
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_UsingFAST_Smb2


On 03/01/18 20:18, Andreas Schneider wrote:

> On Tuesday, 2 January 2018 22:05:43 CET joeg--- via samba-technical wrote:
>> Hi everyone,
> Hi Joe,
>
>> Happy new year!
> same to you!
>
>> I've done some job to use Ansible and OpenStack Cloud to automate the
>> testing of Samba with Windows Protocol Test Suites
>> <https://github.com/Microsoft/WindowsProtocolTestSuites>.
>>
>> So far, I've only tried KDC category, but the other ones are easy to
>> support.
>>
>> This is the current KDC test result:
>>
>> total: 85
>>
>> inconclusive: 3
>>
>> passed: 32
>>
>> failed: 50
> Could please you also run this with MIT Kerberos?
>
>
> Andreas
>

--
Joe Guo
[hidden email]
Catalyst IT

Reply | Threaded
Open this post in threaded view
|

Re: [Report]Automated testing of Samba with Windows Protocol Test Suites

Samba - samba-technical mailing list
On Wednesday, 3 January 2018 21:33:05 CET joeg--- via samba-technical wrote:
> Hi Andreas:
>
> This is the result for Kerberos test suite:

Hi Joe,

so we have MIT Kerberos:
 
> total: 168
>
> inconclusive: 5
>
> passed: 37
>
> failed: 126

Heimdal Kerberos:

> >> This is the current KDC test result:
> >>
> >> total: 85
> >>
> >> inconclusive: 3
> >>
> >> passed: 32
> >>
> >> failed: 50

Looks like with MIT Kerberos five tests more are passing. Also do you know why
with MIT Kerberos it has 168 tests scheduled and with Heimdal just 85?



        Andreas




--
Andreas Schneider                   GPG-ID: CC014E3D
Samba Team                             [hidden email]
www.samba.org



Reply | Threaded
Open this post in threaded view
|

Re: [Report]Automated testing of Samba with Windows Protocol Test Suites

Samba - samba-technical mailing list
Hi Andreas:

About the tests count, my first result(total 85) is for the KDC category
in Kerberos test suite,

the second result(total 168) is for the whole Kerberos test suite, which
contains a list  of categories including KDC.

So the first one is a subset of the second one.


You can refer to the source code to find out how Microsoft organise them.

Here is the list of test suites:

https://github.com/Microsoft/WindowsProtocolTestSuites/tree/staging/TestSuites

Here is the definition of categories in Kerberos:

https://github.com/Microsoft/WindowsProtocolTestSuites/blob/staging/TestSuites/Kerberos/src/Adapter/TestCategories.cs

And, for each test, it can be grouped into multiple categories, here is
an example:

https://github.com/Microsoft/WindowsProtocolTestSuites/blob/staging/TestSuites/Kerberos/src/TestSuite/KILE/KileCrossRealmTest.cs#L37

Hope this answers your question.


On 05/01/18 00:18, Andreas Schneider wrote:

> On Wednesday, 3 January 2018 21:33:05 CET joeg--- via samba-technical wrote:
>> Hi Andreas:
>>
>> This is the result for Kerberos test suite:
> Hi Joe,
>
> so we have MIT Kerberos:
>  
>> total: 168
>>
>> inconclusive: 5
>>
>> passed: 37
>>
>> failed: 126
> Heimdal Kerberos:
>
>>>> This is the current KDC test result:
>>>>
>>>> total: 85
>>>>
>>>> inconclusive: 3
>>>>
>>>> passed: 32
>>>>
>>>> failed: 50
> Looks like with MIT Kerberos five tests more are passing. Also do you know why
> with MIT Kerberos it has 168 tests scheduled and with Heimdal just 85?
>
>
>
> Andreas
>
>
>
>

--
Joe Guo
[hidden email]
Catalyst IT

Reply | Threaded
Open this post in threaded view
|

Re: [Report]Automated testing of Samba with Windows Protocol Test Suites

Samba - samba-technical mailing list
Hi Andreas,

Just talked with Douglas, I realised I misunderstood your question.

Please ignore this email, and I will rebuild samba with MIT Kerberos
Support and re-run the KDC tests.

Thank you!


On 05/01/18 09:41, [hidden email] wrote:

>
> Hi Andreas:
>
> About the tests count, my first result(total 85) is for the KDC
> category in Kerberos test suite,
>
> the second result(total 168) is for the whole Kerberos test suite,
> which contains a list  of categories including KDC.
>
> So the first one is a subset of the second one.
>
>
> You can refer to the source code to find out how Microsoft organise them.
>
> Here is the list of test suites:
>
> https://github.com/Microsoft/WindowsProtocolTestSuites/tree/staging/TestSuites
>
> Here is the definition of categories in Kerberos:
>
> https://github.com/Microsoft/WindowsProtocolTestSuites/blob/staging/TestSuites/Kerberos/src/Adapter/TestCategories.cs
>
> And, for each test, it can be grouped into multiple categories, here
> is an example:
>
> https://github.com/Microsoft/WindowsProtocolTestSuites/blob/staging/TestSuites/Kerberos/src/TestSuite/KILE/KileCrossRealmTest.cs#L37
>
> Hope this answers your question.
>
>
> On 05/01/18 00:18, Andreas Schneider wrote:
>> On Wednesday, 3 January 2018 21:33:05 CET joeg--- via samba-technical wrote:
>>> Hi Andreas:
>>>
>>> This is the result for Kerberos test suite:
>> Hi Joe,
>>
>> so we have MIT Kerberos:
>>  
>>> total: 168
>>>
>>> inconclusive: 5
>>>
>>> passed: 37
>>>
>>> failed: 126
>> Heimdal Kerberos:
>>
>>>>> This is the current KDC test result:
>>>>>
>>>>> total: 85
>>>>>
>>>>> inconclusive: 3
>>>>>
>>>>> passed: 32
>>>>>
>>>>> failed: 50
>> Looks like with MIT Kerberos five tests more are passing. Also do you know why
>> with MIT Kerberos it has 168 tests scheduled and with Heimdal just 85?
>>
>>
>>
>> Andreas
>>
>>
>>
>>
>
> --
> Joe Guo
> [hidden email]
> Catalyst IT

--
Joe Guo
[hidden email]
Catalyst IT

Reply | Threaded
Open this post in threaded view
|

Re: [Report]Automated testing of Samba with Windows Protocol Test Suites

Samba - samba-technical mailing list
In reply to this post by Samba - samba-technical mailing list
Hi Andreas,

I rebuilt samba with MIT Kerberos support (./configure.developer
--with-system-mitkrb5).

The new result is:

total: 85

inconclusive: 2

passed: 4

failed: 79

Failed tests:

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.CanonicalizeSpnInReferralTgt

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileCrossRealmTest.UserInAnotherDomain

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.CaseInsensitive

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.ComputerAccount_DES_PreAuthentication_Fail

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.DesDowngradeProtection

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.ForwardableAndForwardedFlag

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.HwAuthentFlag

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.InitialAndPreAuthentFlag

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.Krbtgt_DES_PreAuthentication_Fail

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.MayPostdateAndPostDatedFlag

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.OkAsDelegateFlag

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.RenewableFlag

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UDPtoTCP

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UnknownPaType

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccount_DES_PreAuthentication_Fail

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountDisabled

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountExpired

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountLocked

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountOutOfLogonHours

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountPasswordMustChangePast

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserAccountPasswordMustChangeZero

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserDelegationNotAllowed

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserPreAuthenticationNotRequired

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserTrustedForDelegation

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UserWithoutUPN

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UsingPasswordWithAES128

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UsingPasswordWithAES256

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UsingPasswordWithDES

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileInteractiveLogonTest.UsingPasswordWithRC4HMAC

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadIntegrity

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadMatch

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadOptions

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorBadProtocolVersionNum

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorCannotPostdate

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorClientPrincipalUnknown

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorETypeNoSupp

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorFieldTooLong

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorGeneric

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorMsgType

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorPreauthFailed

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorRepeat

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorServicePrincipalUnknown

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorSkew

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTicketNeverValid

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTktExpired

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KILE.KileKrbErrorTest.KrbErrorTktNotYetValid

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.AdFxFastArmorInAuthenticator

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.AdFxFastUsedInAuthenticator

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.FastKrb_Error
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.FastOptions_HideClientNames

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.KrbFastFinishedAsRep

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.KrbFastFinishedTgsRep

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.PaFxFastAdvertise

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.StrengthenKey
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileFastTest.UnsupportedFastArmorType

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileSiloTest.Protected_Users_Interactive_Logon_Computer_A2A2_Fail

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileSiloTest.Protected_Users_Interactive_Logon_Computer_A2A2_Succeed

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.InteractiveLogon

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.InteractiveLogonUseExplicitFast

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.KileTestSuite.InteractiveLogonUseFast

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.APDS_KERBEROS_PAC_VALIDATION

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.DomainLocalGroupMembershipWithDisableResourceSIDCompressionSet

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.DomainLocalGroupMembershipWithDisableResourceSIDCompressionUnset

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.KdcSignature
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.KERB_VALIDATION_INFO

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.PAC_CLIENT_INFO

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.PacMarkedAdIfRelevant

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.Request_no_PAC_TGT

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.Request_PAC_TGT

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.ServerSignature

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.PacTestSuite.UPN_DNS_INFO
Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_AdFxFastUsedInAuthenticator

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_FastKrb_Error

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_InteractiveLogonUseExplicitFast

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_InteractiveLogonUseFast

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_KrbFastFinishedAsRep

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_KrbFastFinishedTgsRep

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_StrengthenKey

Microsoft.Protocol.TestSuites.Kerberos.TestSuite.Rc4EncryptTest.RC4_UnsupportedFastArmorType




On 03/01/18 20:18, Andreas Schneider wrote:

> On Tuesday, 2 January 2018 22:05:43 CET joeg--- via samba-technical wrote:
>> Hi everyone,
> Hi Joe,
>
>> Happy new year!
> same to you!
>
>> I've done some job to use Ansible and OpenStack Cloud to automate the
>> testing of Samba with Windows Protocol Test Suites
>> <https://github.com/Microsoft/WindowsProtocolTestSuites>.
>>
>> So far, I've only tried KDC category, but the other ones are easy to
>> support.
>>
>> This is the current KDC test result:
>>
>> total: 85
>>
>> inconclusive: 3
>>
>> passed: 32
>>
>> failed: 50
> Could please you also run this with MIT Kerberos?
>
>
> Andreas
>

--
Joe Guo
[hidden email]
Catalyst IT

Reply | Threaded
Open this post in threaded view
|

Re: [Report]Automated testing of Samba with Windows Protocol Test Suites

Samba - samba-technical mailing list
On Friday, 5 January 2018 01:15:02 CET [hidden email] wrote:

> Hi Andreas,
>
> I rebuilt samba with MIT Kerberos support (./configure.developer
> --with-system-mitkrb5).
>
> The new result is:
>
> total: 85
>
> inconclusive: 2
>
> passed: 4

That looks strange, when I was at Microsoft I had a lot more tests passing.

https://twitter.com/cryptomilk/status/781269063377031170


        Andreas

--
Andreas Schneider                   GPG-ID: CC014E3D
Samba Team                             [hidden email]
www.samba.org



Reply | Threaded
Open this post in threaded view
|

Re: [Report]Automated testing of Samba with Windows Protocol Test Suites

Samba - samba-technical mailing list
Hi Andreas,

Thank you for providing your own test result, very helpful.

I will check my scripts to find out what is wrong.


On 05/01/18 20:00, Andreas Schneider wrote:

> On Friday, 5 January 2018 01:15:02 CET [hidden email] wrote:
>> Hi Andreas,
>>
>> I rebuilt samba with MIT Kerberos support (./configure.developer
>> --with-system-mitkrb5).
>>
>> The new result is:
>>
>> total: 85
>>
>> inconclusive: 2
>>
>> passed: 4
> That looks strange, when I was at Microsoft I had a lot more tests passing.
>
> https://twitter.com/cryptomilk/status/781269063377031170
>
>
> Andreas
>

--
Joe Guo
[hidden email]
Catalyst IT

Reply | Threaded
Open this post in threaded view
|

Re: [Report]Automated testing of Samba with Windows Protocol Test Suites

Samba - samba-technical mailing list
In reply to this post by Samba - samba-technical mailing list
Hi Andreas,

With Andrew Bartlett's help, I find out that my configure option was wrong:

I was using:

./configure.developer --with-system-mitkrb5

which should be:

 ./configure.developer --with-system-mitkrb5 /usr/local/
--with-system-mitkdc /usr/local/sbin/krb5kdc

For me, the Kerberos suite test result with MIT Kerberos support is:

total: 168
inconclusive: 4
passed: 21
failed: 143

And to compare, the test result with default Heimdal Kerberos support is:

total: 168
inconclusive: 5
passed: 37
failed: 126


On 05/01/18 20:00, Andreas Schneider wrote:

> On Friday, 5 January 2018 01:15:02 CET [hidden email] wrote:
>> Hi Andreas,
>>
>> I rebuilt samba with MIT Kerberos support (./configure.developer
>> --with-system-mitkrb5).
>>
>> The new result is:
>>
>> total: 85
>>
>> inconclusive: 2
>>
>> passed: 4
> That looks strange, when I was at Microsoft I had a lot more tests passing.
>
> https://twitter.com/cryptomilk/status/781269063377031170
>
>
> Andreas
>

--
Joe Guo
[hidden email]
Catalyst IT

Reply | Threaded
Open this post in threaded view
|

Re: [Report]Automated testing of Samba with Windows Protocol Test Suites

Samba - samba-technical mailing list
On Monday, 8 January 2018 22:48:52 CET [hidden email] wrote:

> Hi Andreas,
>
> With Andrew Bartlett's help, I find out that my configure option was wrong:
>
> I was using:
>
> ./configure.developer --with-system-mitkrb5
>
> which should be:
>
>  ./configure.developer --with-system-mitkrb5 /usr/local/
> --with-system-mitkdc /usr/local/sbin/krb5kdc
>
> For me, the Kerberos suite test result with MIT Kerberos support is:
>
> total: 168
> inconclusive: 4
> passed: 21
> failed: 143

Thanks,

I think the issue is that the kdc segfaults at one point. I have a patch for
it, but I didn't investigate what the exact issue is.

https://git.samba.org/?p=asn/
samba.git;a=commitdiff;h=32969c3fd93773db27ba377c765554ace34c2879

FAST is not supported yet, and I would need the time to implement the missing
features in MIT Kerberos first.


        Andreas

--
Andreas Schneider                   GPG-ID: CC014E3D
Samba Team                             [hidden email]
www.samba.org