[stable 9] broken hwpstate calls
Locked
 
|
|
|
|
|
Hello,
a user at BSDForen.de had the same problem and I helped him to track it down. While I was unable to find a solution I found the cause of the problems. The problem is (all files and line numbers relative to FreeBSD 9.0-RELEASE): 1. When a new p-state is requested (by powerd or by changing the sysctl) in kern/kern_cpu.c the function cf_set_method() is invoked. 2. In line 335 the driver depended function is called. For newer AMD CPU it's hwpstate_set() in x86/cpufreq/hwpstate.c 3. In x86/cpufreq/hwpstate.c line 227 hwpstate_goto_pstate() is called which does the actual magic. 4. In line 199 "if (msr != id)" triggers, returns ENXIO. The error is send back to cf_set_method(), which prints the "hwpstate0: set freq failed". powerd translates the error to "Device not configured" After some further investigation and looking at the linux driver [0] I changed the loop at x86/cpufreq/hwpstate.c 188ff from 100 iterations to 250. It lessend the problem but didn't solve it. The next step was to rewrite the logic between line 183 and 203 and adding a lot of debug printf. The patch (non style(9) compliant) is attached. With the new logic every 100 usec the new p-state is set again, until it's accepted. After 100 tries ENXIO is returned. This lessend the problem even more and showed that - On an old Phenom II X4 940 (K10 / Deneb) the new p-state is always accepted at the first try. At a test run for about 3 hours there was not a single failure. - On the Bulldozer CPU in about 9 in 10 times the new p-state is accepted at the first try. At most other times the new p-state is accepted after 1 to 10 tries. And there is a ~0,25% chance that the new p-state is never accepted, leading to "hwpstate0: set freq failed". At the next call to cf_set_method() (about 500ms to 1s later) the new p-state it's most likely set successfull. This can be seen at the log (full log attached): # First call, failed after 102 iterations hwpstate0: MSR: 0 ID: 1 hwpstate0: Setting failed! hwpstate0: Iterations: 102 # Second call, successfull hwpstate0: setting P1-state on cpu1 hwpstate0: MSR: 1 ID: 1 hwpstate0: Iterations: 1 So the big question is: Why is the new p-state sometimes not accepted? And why does this only happen on Bulldozer CPUs and not at the old K10 (Barcelona, Deneb), etc? Reading the "BIOS and kernel developer guide" for Bulldozer didn't show anything, but I may have overlooked it. One solution may be to change hwpstate not to set p-states but "Frequency IDs" (FID) and "Voltage ID" (VID) like the linux driver does. 0: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=blob;f=drivers/cpufreq/powernow-k8.c;h=c0e816468e300f242735f4825d09b9d291a9b522;hb=HEAD 1: http://support.amd.com/us/Processor_TechDocs/42301_15h_Mod_00h-0Fh_BKDG.pdf -- Homepage: www.yamagi.org XMPP: [hidden email] GnuPG/GPG: 0xEFBCCBCB  hwpstate.patch (2K) Download Attachment hwpstate.log (87K) Download Attachment attachment2 (203 bytes) Download Attachment |
|
|
|
|
|
|
|
|
|
In reply to this post by Jung-uk Kim
On Fri, 25 May 2012 16:05:56 -0400
Jung-uk Kim <[hidden email]> wrote: > > if we decide so, then I think that we could still keep the things > > "simple". As we currently use the "wholesale" approach (all CPUs > > are set to the same P-state regardless of topology), then we could > > first make a pass of writing the MSR on all processors with a new > > P-state value and then make another pass of checking via MSR > > C001_0063 that the P-state is acquired. > > No, I believe checking MSRC001_0071[18:16] is much simpler if it > works. And it does not break current cpufreq(4) design principles. Okay, thank's for your input. I'll come up with a patch. But it won't happen until tuesday or wednesday next week. -- Homepage: www.yamagi.org XMPP: [hidden email] GnuPG/GPG: 0xEFBCCBCB |
|
|
On Sat, 26 May 2012 12:34:25 +0300
Andriy Gapon <[hidden email]> wrote: > >>> if we decide so, then I think that we could still keep the things > >>> "simple". As we currently use the "wholesale" approach (all CPUs are > >>> set to the same P-state regardless of topology), then we could first > >>> make a pass of writing the MSR on all processors with a new P-state > >>> value and then make another pass of checking via MSR C001_0063 that the > >>> P-state is acquired. > >> > >> No, I believe checking MSRC001_0071[18:16] is much simpler if it works. > >> And it does not break current cpufreq(4) design principles. One potential problem with MSRC001_0071[18:16] is, that it's on older CPUs supported by hwpstate it's the same as C001_0063. Only on newer models with "turbo" it containts the actual "hardware p-state". So additional logic would be required: 1. Set new p-state 2. Check CPUID for support of "hardware p-states" 3.1 If yes, read MSRC001_0071[18:16] and convert the "hardware p-state" into a "software p-state" 3.2 If not, just read MSRC001_0071[18:16] 4. Compare read (and converted) p-state to the requested p-state I don't think that it's worth this additional efford. The solution suggest by Andriy Gapon is trivial, works fine and is supported by all CPUs supported by hwpstate. > I believe the approach that I suggested to be so trivial to implement (and also > correct) that here is a patch: > > diff --git a/sys/x86/cpufreq/hwpstate.c b/sys/x86/cpufreq/hwpstate.c > index 40e1943..9c17a41 100644 > --- a/sys/x86/cpufreq/hwpstate.c > +++ b/sys/x86/cpufreq/hwpstate.c > @@ -186,16 +186,21 @@ hwpstate_goto_pstate(device_t dev, int pstate) > id, PCPU_GET(cpuid)); > /* Go To Px-state */ > wrmsr(MSR_AMD_10H_11H_CONTROL, id); > + } > + CPU_FOREACH(i) { > + /* Bind to each cpu. */ > + thread_lock(curthread); > + sched_bind(curthread, i); > + thread_unlock(curthread); > /* wait loop (100*100 usec is enough ?) */ > for(j = 0; j < 100; j++){ > + /* get the result. not assure msr=id */ > msr = rdmsr(MSR_AMD_10H_11H_STATUS); > if(msr == id){ > break; > } > DELAY(100); > } > - /* get the result. not assure msr=id */ > - msr = rdmsr(MSR_AMD_10H_11H_STATUS); > HWPSTATE_DEBUG(dev, "result P%d-state on cpu%d\n", > (int)msr, PCPU_GET(cpuid)); > if (msr != id) { Phenom II "Deneb". -- Homepage: www.yamagi.org XMPP: [hidden email] GnuPG/GPG: 0xEFBCCBCB |
|
|
|
|
|
|
|
|
|
|
|
Loading...
| Powered by Nabble | Edit this page |