Discussion:
[opensuse-m17n] Konsole displays some characters wrong
Norman Diamond
2015-06-09 02:41:54 UTC
Permalink
In openSUSE 13.2, in Konsole, downarrow↓displays↓wrong.
The file encoding, Konsole terminal's encoding, and vim's encoding are all UTF-8.
Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three.
vim counts characters properly but the display is wrong.
more, cat, etc., display wrong.

I used Suse 11.0 for six years.  Konsole had no problem with downarrows.

Yours sincerely,
Norman Diamond
--
To unsubscribe, e-mail: opensuse-m17n+***@opensuse.org
To contact the owner, e-mail: opensuse-m17n+***@opensuse.org
Fuminobu TAKEYAMA
2015-06-10 10:43:11 UTC
Permalink
Hi,

Which font do you use on Konsole?

Then, can you share the result of fc-match -v?
If the font is "Monospace" then run the following command:
$ fc-match -v Monospace

# you can snip off "charset:" section


Fuminobu TAKEYAMA
Post by Norman Diamond
In openSUSE 13.2, in Konsole, downarrow↓displays↓wrong.
The file encoding, Konsole terminal's encoding, and vim's encoding are all UTF-8.
Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three.
vim counts characters properly but the display is wrong.
more, cat, etc., display wrong.
I used Suse 11.0 for six years. Konsole had no problem with downarrows.
Yours sincerely,
Norman Diamond
--
To unsubscribe, e-mail: opensuse-m17n+***@opensuse.org
To contact the owner, e-mail: opensuse-m17n+***@opensuse.org
Norman Diamond
2015-06-10 21:42:39 UTC
Permalink
Yes the font is Monospace.  I did not change it from the default.  A Konsole settings panel displayed the name Monospace and allowed me to change the font size but didn't allow me to select a different font.

For this e-mail I use Yahoo's web interface from Internet Explorer under Windows, so I translated the output of fc-match -v and fc-match -v Monospace from UTF-8 to Microsoft's Unicode encoding.  Yahoo's web interface will scramble the text that I will paste below, so I ALSO attach the outputs as files.


Output of fc-match -v

Pattern has 41 elts (size 48)
 family: "IPA Pゎシック"(s) "IPAPGothic"(s)
 familylang: "ja"(s) "en"(s)
 style: "Regular"(w)
 stylelang: "en"(w)
 fullname: "IPAPGothic"(w) "IPA Pゎシック"(w)
 fullnamelang: "en"(w) "ja"(w)
 slant: 0(i)(s)
 weight: 80(i)(s)
 width: 100(i)(s)
 size: 12(f)(s)
 pixelsize: 12.5(f)(s)
 foundry: "unknown"(w)
 antialias: True(w)
 hintstyle: 0(i)(w)
 hinting: True(w)
 verticallayout: False(s)
 autohint: True(w)
 globaladvance: True(s)
 file: "/usr/share/fonts/truetype/ipagp.ttf"(w)
 index: 0(i)(w)
 outline: True(w)
 scalable: True(w)
 dpi: 75(f)(s)
 rgba: 5(i)(w)
 scale: 1(f)(s)
 charset:
 [snip]
(w)
 lang: aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
 fontversion: 198574(i)(s)
 capability: "otlayout:hani otlayout:kana otlayout:latn"(w)
 fontformat: "TrueType"(w)
 embeddedbitmap: True(w)
 decorative: False(s)
 lcdfilter: 0(i)(w)
 namelang: "ja"(s)
 prgname: "fc-match"(s)
 hash: "sha256:a63f6153841e56ec9df1b31a54d98d3f41d8aaa1c3d8df30a33502296f5b8068"(w)
 postscriptname: "IPAPGothic"(w)
 force_hintstyle: "none"(w)
 force_autohint: False(w)
 force_bw: False(w)
 force_bw_monospace: False(w)

Output of fc-match -v Monospace

Pattern has 42 elts (size 48)
 family: "IPAゎシック"(s) "IPAGothic"(s)
 familylang: "ja"(s) "en"(s)
 style: "Regular"(w)
 stylelang: "en"(w)
 fullname: "IPAGothic"(w) "IPAゎシック"(w)
 fullnamelang: "en"(w) "ja"(w)
 slant: 0(i)(s)
 weight: 80(i)(s)
 width: 100(i)(s)
 size: 12(f)(s)
 pixelsize: 12.5(f)(s)
 spacing: 90(i)(w)
 foundry: "unknown"(w)
 antialias: True(w)
 hintstyle: 0(i)(w)
 hinting: True(w)
 verticallayout: False(s)
 autohint: True(w)
 globaladvance: True(s)
 file: "/usr/share/fonts/truetype/ipag.ttf"(w)
 index: 0(i)(w)
 outline: True(w)
 scalable: True(w)
 dpi: 75(f)(s)
 rgba: 5(i)(w)
 scale: 1(f)(s)
 charset:
 [snip]
(w)
 lang: aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
 fontversion: 198574(i)(s)
 capability: "otlayout:hani otlayout:kana otlayout:latn"(w)
 fontformat: "TrueType"(w)
 embeddedbitmap: False(w)
 decorative: False(s)
 lcdfilter: 0(i)(w)
 namelang: "ja"(s)
 prgname: "fc-match"(s)
 hash: "sha256:503af4a8b84d1079b8e2e358dc7f7a7fb8cb7a1f212f35eaef6782dbfc75a55e"(w)
 postscriptname: "IPAGothic"(w)
 force_hintstyle: "none"(w)
 force_autohint: False(w)
 force_bw: False(w)
 force_bw_monospace: False(w)
----- Original Message -----
Date: 2015/6/10, Wed 19:43
Subject: Re: [opensuse-m17n] Konsole displays some characters wrong
Hi,
Which font do you use on Konsole?
Then, can you share the result of fc-match -v?
$ fc-match -v Monospace
# you can snip off "charset:" section
Fuminobu TAKEYAMA
  In openSUSE 13.2, in Konsole, downarrow↓displays↓wrong.
  The file encoding, Konsole terminal's encoding, and vim's
encoding
are all UTF-8.
  Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three.
  vim counts characters properly but the display is wrong.
  more, cat, etc., display wrong.
  I used Suse 11.0 for six years.  Konsole had no problem with
downarrows.
  Yours sincerely,
  Norman Diamond
--
 
       
Fuminobu TAKEYAMA
2015-06-11 13:46:13 UTC
Permalink
Ok, Monospace is an alias of IPAGothic on your environment.

The font, IPAGothic provides double-width down-allow glyph
although Konsole assumes that the glyph should have the same width as
glyphs such as 'a' and '0'.

A solution is to use another font such as DejaVu Sans Mono.
You can change fonts from:
Settings > Edit Current Profile... > Appearance

If you do not need Japanese fonts, please uninstall ipa-gothic-fonts.
So another font is to be selected for Monospace.


Fuminobu TAKEYAMA
Yes the font is Monospace. I did not change it from the default. A Konsole settings panel displayed the name Monospace and allowed me to change the font size but didn't allow me to select a different font.
For this e-mail I use Yahoo's web interface from Internet Explorer under Windows, so I translated the output of fc-match -v and fc-match -v Monospace from UTF-8 to Microsoft's Unicode encoding. Yahoo's web interface will scramble the text that I will paste below, so I ALSO attach the outputs as files.
Output of fc-match -v
Pattern has 41 elts (size 48)
family: "IPA Pゴシック"(s) "IPAPGothic"(s)
familylang: "ja"(s) "en"(s)
style: "Regular"(w)
stylelang: "en"(w)
fullname: "IPAPGothic"(w) "IPA Pゴシック"(w)
fullnamelang: "en"(w) "ja"(w)
slant: 0(i)(s)
weight: 80(i)(s)
width: 100(i)(s)
size: 12(f)(s)
pixelsize: 12.5(f)(s)
foundry: "unknown"(w)
antialias: True(w)
hintstyle: 0(i)(w)
hinting: True(w)
verticallayout: False(s)
autohint: True(w)
globaladvance: True(s)
file: "/usr/share/fonts/truetype/ipagp.ttf"(w)
index: 0(i)(w)
outline: True(w)
scalable: True(w)
dpi: 75(f)(s)
rgba: 5(i)(w)
scale: 1(f)(s)
[snip]
(w)
lang: aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
fontversion: 198574(i)(s)
capability: "otlayout:hani otlayout:kana otlayout:latn"(w)
fontformat: "TrueType"(w)
embeddedbitmap: True(w)
decorative: False(s)
lcdfilter: 0(i)(w)
namelang: "ja"(s)
prgname: "fc-match"(s)
hash: "sha256:a63f6153841e56ec9df1b31a54d98d3f41d8aaa1c3d8df30a33502296f5b8068"(w)
postscriptname: "IPAPGothic"(w)
force_hintstyle: "none"(w)
force_autohint: False(w)
force_bw: False(w)
force_bw_monospace: False(w)
Output of fc-match -v Monospace
Pattern has 42 elts (size 48)
family: "IPAゴシック"(s) "IPAGothic"(s)
familylang: "ja"(s) "en"(s)
style: "Regular"(w)
stylelang: "en"(w)
fullname: "IPAGothic"(w) "IPAゴシック"(w)
fullnamelang: "en"(w) "ja"(w)
slant: 0(i)(s)
weight: 80(i)(s)
width: 100(i)(s)
size: 12(f)(s)
pixelsize: 12.5(f)(s)
spacing: 90(i)(w)
foundry: "unknown"(w)
antialias: True(w)
hintstyle: 0(i)(w)
hinting: True(w)
verticallayout: False(s)
autohint: True(w)
globaladvance: True(s)
file: "/usr/share/fonts/truetype/ipag.ttf"(w)
index: 0(i)(w)
outline: True(w)
scalable: True(w)
dpi: 75(f)(s)
rgba: 5(i)(w)
scale: 1(f)(s)
[snip]
(w)
lang: aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
fontversion: 198574(i)(s)
capability: "otlayout:hani otlayout:kana otlayout:latn"(w)
fontformat: "TrueType"(w)
embeddedbitmap: False(w)
decorative: False(s)
lcdfilter: 0(i)(w)
namelang: "ja"(s)
prgname: "fc-match"(s)
hash: "sha256:503af4a8b84d1079b8e2e358dc7f7a7fb8cb7a1f212f35eaef6782dbfc75a55e"(w)
postscriptname: "IPAGothic"(w)
force_hintstyle: "none"(w)
force_autohint: False(w)
force_bw: False(w)
force_bw_monospace: False(w)
----- Original Message -----
Date: 2015/6/10, Wed 19:43
Subject: Re: [opensuse-m17n] Konsole displays some characters wrong
Hi,
Which font do you use on Konsole?
Then, can you share the result of fc-match -v?
$ fc-match -v Monospace
# you can snip off "charset:" section
Fuminobu TAKEYAMA
Post by Norman Diamond
In openSUSE 13.2, in Konsole, downarrow↓displays↓wrong.
The file encoding, Konsole terminal's encoding, and vim's
encoding
are all UTF-8.
Post by Norman Diamond
Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three.
vim counts characters properly but the display is wrong.
more, cat, etc., display wrong.
I used Suse 11.0 for six years. Konsole had no problem with
downarrows.
Post by Norman Diamond
Yours sincerely,
Norman Diamond
--
--
To unsubscribe, e-mail: opensuse-m17n+***@opensuse.org
To contact the owner, e-mail: opensuse-m17n+***@opensuse.org
Norman Diamond
2015-06-11 21:48:46 UTC
Permalink
I click 設定 > 現在のプロファイルを編集(E)... > 外観
In フォント > プレビュー it says Monospace
Oh now I see, next to フォントサイズ where it lets me change the size, there's a clickable button to select a font.
In that panel it says モノスペース instead of Monospace.

Thank you for the workaround.  I will experiment with some fonts.  I would like to recommend that the preview box should include some full-width characters so that users can judge more easily.  If a font doesn't include glyphs for full-width characters then users will see immediately that they shouldn't experiment with such a font.

I still think there's a bug.

Konsole knows that the glyph for 字 is full-width, so Konsole should know that the glyph for downarrow is full-width.

The version of Konsole that was included in Suse 11.0 did know that downarrow is full-width.  I never had to look at what the default font was because it just worked.

Yours sincerely,
Norman Diamond
Post by Fuminobu TAKEYAMA
Ok, Monospace is an alias of IPAGothic on your environment.
The font, IPAGothic provides double-width down-allow glyph
although Konsole assumes that the glyph should have the same width as
glyphs such as 'a' and '0'.
A solution is to use another font such as DejaVu Sans Mono.
Settings > Edit Current Profile... > Appearance
If you do not need Japanese fonts, please uninstall ipa-gothic-fonts.
So another font is to be selected for Monospace.
Fuminobu TAKEYAMA
Yes the font is Monospace.  I did not change it from the default.  A
Konsole settings panel displayed the name Monospace and allowed me to change the
font size but didn't allow me to select a different font.
For this e-mail I use Yahoo's web interface from Internet Explorer
under Windows, so I translated the output of fc-match -v and fc-match -v
Monospace from UTF-8 to Microsoft's Unicode encoding.  Yahoo's web
interface will scramble the text that I will paste below, so I ALSO attach the
outputs as files.
Output of fc-match -v
Pattern has 41 elts (size 48)
  family: "IPA Pゴシック"(s) "IPAPGothic"(s)
  familylang: "ja"(s) "en"(s)
  style: "Regular"(w)
  stylelang: "en"(w)
  fullname: "IPAPGothic"(w) "IPA Pゴシック"(w)
  fullnamelang: "en"(w) "ja"(w)
  slant: 0(i)(s)
  weight: 80(i)(s)
  width: 100(i)(s)
  size: 12(f)(s)
  pixelsize: 12.5(f)(s)
  foundry: "unknown"(w)
  antialias: True(w)
  hintstyle: 0(i)(w)
  hinting: True(w)
  verticallayout: False(s)
  autohint: True(w)
  globaladvance: True(s)
  file: "/usr/share/fonts/truetype/ipagp.ttf"(w)
  index: 0(i)(w)
  outline: True(w)
  scalable: True(w)
  dpi: 75(f)(s)
  rgba: 5(i)(w)
  scale: 1(f)(s)
  [snip]
(w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
  fontversion: 198574(i)(s)
  capability: "otlayout:hani otlayout:kana otlayout:latn"(w)
  fontformat: "TrueType"(w)
  embeddedbitmap: True(w)
  decorative: False(s)
  lcdfilter: 0(i)(w)
  namelang: "ja"(s)
  prgname: "fc-match"(s)
"sha256:a63f6153841e56ec9df1b31a54d98d3f41d8aaa1c3d8df30a33502296f5b8068"(w)
  postscriptname: "IPAPGothic"(w)
  force_hintstyle: "none"(w)
  force_autohint: False(w)
  force_bw: False(w)
  force_bw_monospace: False(w)
Output of fc-match -v Monospace
Pattern has 42 elts (size 48)
  family: "IPAゴシック"(s) "IPAGothic"(s)
  familylang: "ja"(s) "en"(s)
  style: "Regular"(w)
  stylelang: "en"(w)
  fullname: "IPAGothic"(w) "IPAゴシック"(w)
  fullnamelang: "en"(w) "ja"(w)
  slant: 0(i)(s)
  weight: 80(i)(s)
  width: 100(i)(s)
  size: 12(f)(s)
  pixelsize: 12.5(f)(s)
  spacing: 90(i)(w)
  foundry: "unknown"(w)
  antialias: True(w)
  hintstyle: 0(i)(w)
  hinting: True(w)
  verticallayout: False(s)
  autohint: True(w)
  globaladvance: True(s)
  file: "/usr/share/fonts/truetype/ipag.ttf"(w)
  index: 0(i)(w)
  outline: True(w)
  scalable: True(w)
  dpi: 75(f)(s)
  rgba: 5(i)(w)
  scale: 1(f)(s)
  [snip]
(w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
  fontversion: 198574(i)(s)
  capability: "otlayout:hani otlayout:kana otlayout:latn"(w)
  fontformat: "TrueType"(w)
  embeddedbitmap: False(w)
  decorative: False(s)
  lcdfilter: 0(i)(w)
  namelang: "ja"(s)
  prgname: "fc-match"(s)
"sha256:503af4a8b84d1079b8e2e358dc7f7a7fb8cb7a1f212f35eaef6782dbfc75a55e"(w)
  postscriptname: "IPAGothic"(w)
  force_hintstyle: "none"(w)
  force_autohint: False(w)
  force_bw: False(w)
  force_bw_monospace: False(w)
----- Original Message -----
  Date: 2015/6/10, Wed 19:43
  Subject: Re: [opensuse-m17n] Konsole displays some characters
wrong
  Hi,
  Which font do you use on Konsole?
  Then, can you share the result of fc-match -v?
  If the font is "Monospace" then run the following
  $ fc-match -v Monospace
  # you can snip off "charset:" section
  Fuminobu TAKEYAMA
    In openSUSE 13.2, in Konsole, downarrow↓displays↓wrong.
    The file encoding, Konsole terminal's encoding, and
vim's
encoding
  are all UTF-8.
    Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three.
    vim counts characters properly but the display is wrong.
    more, cat, etc., display wrong.
    I used Suse 11.0 for six years.  Konsole had no problem with
downarrows.
    Yours sincerely,
    Norman Diamond
  --
--
       
--
To unsubscribe, e-mail: opensuse-m17n+***@opensuse.org
To contact the owner, e-mail: opensuse-m17n+***@opensuse.org
Norman Diamond
2015-06-11 22:05:46 UTC
Permalink
There is a worse bug.

In the 固定幅フォントを選択 - Konsole panel, the bottom part is a text box.
I can type or paste text in the text box.
I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows,
and they display correctly in that text box.
Konsole corrupts the display in its main window, and the preview window does not preview the corruption.

Yours sincerely,Norman Diamond

       
Post by Norman Diamond
I click 設定 > 現在のプロファイルを編集(E)... > 外観
In フォント > プレビュー it says Monospace
Oh now I see, next to フォントサイズ where it lets me change the size, there's a
clickable button to select a font.
In that panel it says モノスペース instead of Monospace.
Thank you for the workaround.  I will experiment with some fonts.  I would like
to recommend that the preview box should include some full-width characters so
that users can judge more easily.  If a font doesn't include glyphs for
full-width characters then users will see immediately that they shouldn't
experiment with such a font.
I still think there's a bug.
Konsole knows that the glyph for 字 is full-width, so Konsole should know that
the glyph for downarrow is full-width.
The version of Konsole that was included in Suse 11.0 did know that downarrow is
full-width.  I never had to look at what the default font was because it just
worked.
Yours sincerely,
Norman Diamond
Post by Fuminobu TAKEYAMA
Ok, Monospace is an alias of IPAGothic on your environment.
The font, IPAGothic provides double-width down-allow glyph
although Konsole assumes that the glyph should have the same width as
glyphs such as 'a' and '0'.
A solution is to use another font such as DejaVu Sans Mono.
Settings > Edit Current Profile... > Appearance
If you do not need Japanese fonts, please uninstall ipa-gothic-fonts.
So another font is to be selected for Monospace.
Fuminobu TAKEYAMA
  Yes the font is Monospace.  I did not change it from the default.  A
Konsole settings panel displayed the name Monospace and allowed me to
change the
Post by Fuminobu TAKEYAMA
font size but didn't allow me to select a different font.
  For this e-mail I use Yahoo's web interface from Internet Explorer
under Windows, so I translated the output of fc-match -v and fc-match -v
Monospace from UTF-8 to Microsoft's Unicode encoding.  Yahoo's web
interface will scramble the text that I will paste below, so I ALSO attach
the
Post by Fuminobu TAKEYAMA
outputs as files.
  Output of fc-match -v
  Pattern has 41 elts (size 48)
   family: "IPA Pゴシック"(s) "IPAPGothic"(s)
   familylang: "ja"(s) "en"(s)
   style: "Regular"(w)
   stylelang: "en"(w)
   fullname: "IPAPGothic"(w) "IPA Pゴシック"(w)
   fullnamelang: "en"(w) "ja"(w)
   slant: 0(i)(s)
   weight: 80(i)(s)
   width: 100(i)(s)
   size: 12(f)(s)
   pixelsize: 12.5(f)(s)
   foundry: "unknown"(w)
   antialias: True(w)
   hintstyle: 0(i)(w)
   hinting: True(w)
   verticallayout: False(s)
   autohint: True(w)
   globaladvance: True(s)
   file: "/usr/share/fonts/truetype/ipagp.ttf"(w)
   index: 0(i)(w)
   outline: True(w)
   scalable: True(w)
   dpi: 75(f)(s)
   rgba: 5(i)(w)
   scale: 1(f)(s)
   [snip]
  (w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
Post by Fuminobu TAKEYAMA
   fontversion: 198574(i)(s)
   capability: "otlayout:hani otlayout:kana otlayout:latn"(w)
   fontformat: "TrueType"(w)
   embeddedbitmap: True(w)
   decorative: False(s)
   lcdfilter: 0(i)(w)
   namelang: "ja"(s)
   prgname: "fc-match"(s)
"sha256:a63f6153841e56ec9df1b31a54d98d3f41d8aaa1c3d8df30a33502296f5b8068"(w)
Post by Fuminobu TAKEYAMA
   postscriptname: "IPAPGothic"(w)
   force_hintstyle: "none"(w)
   force_autohint: False(w)
   force_bw: False(w)
   force_bw_monospace: False(w)
  Output of fc-match -v Monospace
  Pattern has 42 elts (size 48)
   family: "IPAゴシック"(s) "IPAGothic"(s)
   familylang: "ja"(s) "en"(s)
   style: "Regular"(w)
   stylelang: "en"(w)
   fullname: "IPAGothic"(w) "IPAゴシック"(w)
   fullnamelang: "en"(w) "ja"(w)
   slant: 0(i)(s)
   weight: 80(i)(s)
   width: 100(i)(s)
   size: 12(f)(s)
   pixelsize: 12.5(f)(s)
   spacing: 90(i)(w)
   foundry: "unknown"(w)
   antialias: True(w)
   hintstyle: 0(i)(w)
   hinting: True(w)
   verticallayout: False(s)
   autohint: True(w)
   globaladvance: True(s)
   file: "/usr/share/fonts/truetype/ipag.ttf"(w)
   index: 0(i)(w)
   outline: True(w)
   scalable: True(w)
   dpi: 75(f)(s)
   rgba: 5(i)(w)
   scale: 1(f)(s)
   [snip]
  (w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
Post by Fuminobu TAKEYAMA
   fontversion: 198574(i)(s)
   capability: "otlayout:hani otlayout:kana otlayout:latn"(w)
   fontformat: "TrueType"(w)
   embeddedbitmap: False(w)
   decorative: False(s)
   lcdfilter: 0(i)(w)
   namelang: "ja"(s)
   prgname: "fc-match"(s)
"sha256:503af4a8b84d1079b8e2e358dc7f7a7fb8cb7a1f212f35eaef6782dbfc75a55e"(w)
Post by Fuminobu TAKEYAMA
   postscriptname: "IPAGothic"(w)
   force_hintstyle: "none"(w)
   force_autohint: False(w)
   force_bw: False(w)
   force_bw_monospace: False(w)
  ----- Original Message -----
   Date: 2015/6/10, Wed 19:43
   Subject: Re: [opensuse-m17n] Konsole displays some
characters
Post by Fuminobu TAKEYAMA
wrong
   Hi,
   Which font do you use on Konsole?
   Then, can you share the result of fc-match -v?
   If the font is "Monospace" then run the following
   $ fc-match -v Monospace
   # you can snip off "charset:" section
   Fuminobu TAKEYAMA
    In openSUSE 13.2, in Konsole, downarrow↓displays↓wrong.
    The file encoding, Konsole terminal's encoding, and
vim's
  encoding
   are all UTF-8.
    Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three.
    vim counts characters properly but the display is
wrong.
Post by Fuminobu TAKEYAMA
    more, cat, etc., display wrong.
    I used Suse 11.0 for six years.  Konsole had no problem
with
Post by Fuminobu TAKEYAMA
  downarrows.
    Yours sincerely,
    Norman Diamond
   --
--
       
--
--
To unsubscribe, e-mail: opensuse-m17n+***@opensuse.org
To contact the owner, e-mail: opensuse-m17n+***@opensuse.org
Fuminobu TAKEYAMA
2015-06-12 16:30:25 UTC
Permalink
Post by Norman Diamond
Post by Norman Diamond
I still think there's a bug.
Yes, this is clearly a Japanese specific bug.
Could you file a bug report for openSUSE and KDE?

I saw a patch to fix the same problem with emacs about 5 years ago.
So I think this problem can be resolved by some hacks.
Post by Norman Diamond
Post by Norman Diamond
Konsole knows that the glyph for 字 is full-width, so Konsole should know that
the glyph for downarrow is full-width.
Down arrow is known as ambiguous-width character.
Monospaced font should provide half-width down arrow
but some Japanese fonts such as IPA Gothic and MS Gothic do full-width down arrow
from a kind of historical reason.
Post by Norman Diamond
Post by Norman Diamond
The version of Konsole that was included in Suse 11.0 did know that downarrow is
full-width. I never had to look at what the default font was because it just
worked.
Konsole's character layouting was changed recently as far as I know
so that some (flexible-width) bold fonts can be placed correctly.

I guess it causes this problem.
Post by Norman Diamond
In the 固定幅フォントを選択 - Konsole panel, the bottom part is a text box.
I can type or paste text in the text box.
I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows,
and they display correctly in that text box.
Konsole corrupts the display in its main window, and the preview window does not preview the
corruption.

Unfortunately, the preview is for font selection dialog and Qt widgets.
It is not preview of Konsole.

Maybe, the preview should be removed.


Fuminobu TAKEYAMA
Post by Norman Diamond
There is a worse bug.
In the 固定幅フォントを選択 - Konsole panel, the bottom part is a text box.
I can type or paste text in the text box.
I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows,
and they display correctly in that text box.
Konsole corrupts the display in its main window, and the preview window does not preview the corruption.
Yours sincerely,Norman Diamond
Post by Norman Diamond
I click 設定 > 現在のプロファイルを編集(E)... > 外観
In フォント > プレビュー it says Monospace
Oh now I see, next to フォントサイズ where it lets me change the size, there's a
clickable button to select a font.
In that panel it says モノスペース instead of Monospace.
Thank you for the workaround. I will experiment with some fonts. I would like
to recommend that the preview box should include some full-width characters so
that users can judge more easily. If a font doesn't include glyphs for
full-width characters then users will see immediately that they shouldn't
experiment with such a font.
I still think there's a bug.
Konsole knows that the glyph for 字 is full-width, so Konsole should know that
the glyph for downarrow is full-width.
The version of Konsole that was included in Suse 11.0 did know that downarrow is
full-width. I never had to look at what the default font was because it just
worked.
Yours sincerely,
Norman Diamond
Post by Fuminobu TAKEYAMA
Ok, Monospace is an alias of IPAGothic on your environment.
The font, IPAGothic provides double-width down-allow glyph
although Konsole assumes that the glyph should have the same width as
glyphs such as 'a' and '0'.
A solution is to use another font such as DejaVu Sans Mono.
Settings > Edit Current Profile... > Appearance
If you do not need Japanese fonts, please uninstall ipa-gothic-fonts.
So another font is to be selected for Monospace.
Fuminobu TAKEYAMA
Yes the font is Monospace. I did not change it from the default. A
Konsole settings panel displayed the name Monospace and allowed me to
change the
Post by Fuminobu TAKEYAMA
font size but didn't allow me to select a different font.
For this e-mail I use Yahoo's web interface from Internet Explorer
under Windows, so I translated the output of fc-match -v and fc-match -v
Monospace from UTF-8 to Microsoft's Unicode encoding. Yahoo's web
interface will scramble the text that I will paste below, so I ALSO attach
the
Post by Fuminobu TAKEYAMA
outputs as files.
Output of fc-match -v
Pattern has 41 elts (size 48)
family: "IPA Pゴシック"(s) "IPAPGothic"(s)
familylang: "ja"(s) "en"(s)
style: "Regular"(w)
stylelang: "en"(w)
fullname: "IPAPGothic"(w) "IPA Pゴシック"(w)
fullnamelang: "en"(w) "ja"(w)
slant: 0(i)(s)
weight: 80(i)(s)
width: 100(i)(s)
size: 12(f)(s)
pixelsize: 12.5(f)(s)
foundry: "unknown"(w)
antialias: True(w)
hintstyle: 0(i)(w)
hinting: True(w)
verticallayout: False(s)
autohint: True(w)
globaladvance: True(s)
file: "/usr/share/fonts/truetype/ipagp.ttf"(w)
index: 0(i)(w)
outline: True(w)
scalable: True(w)
dpi: 75(f)(s)
rgba: 5(i)(w)
scale: 1(f)(s)
[snip]
(w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
Post by Fuminobu TAKEYAMA
fontversion: 198574(i)(s)
capability: "otlayout:hani otlayout:kana otlayout:latn"(w)
fontformat: "TrueType"(w)
embeddedbitmap: True(w)
decorative: False(s)
lcdfilter: 0(i)(w)
namelang: "ja"(s)
prgname: "fc-match"(s)
"sha256:a63f6153841e56ec9df1b31a54d98d3f41d8aaa1c3d8df30a33502296f5b8068"(w)
Post by Fuminobu TAKEYAMA
postscriptname: "IPAPGothic"(w)
force_hintstyle: "none"(w)
force_autohint: False(w)
force_bw: False(w)
force_bw_monospace: False(w)
Output of fc-match -v Monospace
Pattern has 42 elts (size 48)
family: "IPAゴシック"(s) "IPAGothic"(s)
familylang: "ja"(s) "en"(s)
style: "Regular"(w)
stylelang: "en"(w)
fullname: "IPAGothic"(w) "IPAゴシック"(w)
fullnamelang: "en"(w) "ja"(w)
slant: 0(i)(s)
weight: 80(i)(s)
width: 100(i)(s)
size: 12(f)(s)
pixelsize: 12.5(f)(s)
spacing: 90(i)(w)
foundry: "unknown"(w)
antialias: True(w)
hintstyle: 0(i)(w)
hinting: True(w)
verticallayout: False(s)
autohint: True(w)
globaladvance: True(s)
file: "/usr/share/fonts/truetype/ipag.ttf"(w)
index: 0(i)(w)
outline: True(w)
scalable: True(w)
dpi: 75(f)(s)
rgba: 5(i)(w)
scale: 1(f)(s)
[snip]
(w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
Post by Fuminobu TAKEYAMA
fontversion: 198574(i)(s)
capability: "otlayout:hani otlayout:kana otlayout:latn"(w)
fontformat: "TrueType"(w)
embeddedbitmap: False(w)
decorative: False(s)
lcdfilter: 0(i)(w)
namelang: "ja"(s)
prgname: "fc-match"(s)
"sha256:503af4a8b84d1079b8e2e358dc7f7a7fb8cb7a1f212f35eaef6782dbfc75a55e"(w)
Post by Fuminobu TAKEYAMA
postscriptname: "IPAGothic"(w)
force_hintstyle: "none"(w)
force_autohint: False(w)
force_bw: False(w)
force_bw_monospace: False(w)
----- Original Message -----
Date: 2015/6/10, Wed 19:43
Subject: Re: [opensuse-m17n] Konsole displays some
characters
Post by Fuminobu TAKEYAMA
wrong
Hi,
Which font do you use on Konsole?
Then, can you share the result of fc-match -v?
If the font is "Monospace" then run the following
$ fc-match -v Monospace
# you can snip off "charset:" section
Fuminobu TAKEYAMA
Post by Norman Diamond
In openSUSE 13.2, in Konsole, downarrow↓displays↓wrong.
The file encoding, Konsole terminal's encoding, and
vim's
encoding
are all UTF-8.
Post by Norman Diamond
Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three.
vim counts characters properly but the display is
wrong.
Post by Fuminobu TAKEYAMA
Post by Norman Diamond
more, cat, etc., display wrong.
I used Suse 11.0 for six years. Konsole had no problem
with
Post by Fuminobu TAKEYAMA
downarrows.
Post by Norman Diamond
Yours sincerely,
Norman Diamond
--
--
--
--
To unsubscribe, e-mail: opensuse-m17n+***@opensuse.org
To contact the owner, e-mail: opensuse-m17n+***@opensuse.org
Norman Diamond
2015-06-13 01:35:03 UTC
Permalink
Since the font selection dialog's preview uses Qt widgets instead of whatever Konsole uses, we know that the full-width downarrow character is displayed as full-width by these Linux tools.  Furthermore in Suse 11.0 Konsole itself also displayed the full-width downarrow character as full-width.  So I think this bug is in the newer version of Konsole itself, not in the rest of Linux.  I can't say if the preview window should be removed because it doesn't really provide a preview, but it helped narrow down this bug.

I do not agree with your statement that downarrow should be displayed as half-width.  It is still the same character that it used to be in ShiftJIS and EUC encodings, and the adoption of Unicode and UTF-8 does not change the width of the glyph.

If I the character becomes half-width when I edit source files in Konsole, but it's still full-width when other people edit the same source files in a Gnome terminal or even in Windows, it will result in people scrambling source files.

Full-width a and half-width a do not have ambiguous widths.  Full-width ア and half-width ア do not have ambiguous widths.  I do not know who decided that downarrow should have an ambiguous width.  If the Unicode consortium decided that, they're causing problems.

Yours sincerely,
Norman Diamond


----- Forwarded Message -----
Date: 2015/6/13, Sat 01:30
Subject: Re: [opensuse-m17n] Konsole displays some characters wrong
  
Post by Norman Diamond
Post by Norman Diamond
I still think there's a bug.
Yes, this is clearly a Japanese specific bug.
Could you file a bug report for openSUSE and KDE?
I saw a patch to fix the same problem with emacs about 5 years ago.
So I think this problem can be resolved by some hacks.
Post by Norman Diamond
Post by Norman Diamond
Konsole knows that the glyph for 字 is full-width, so Konsole should know that
the glyph for downarrow is full-width.
Down arrow is known as ambiguous-width character.
Monospaced font should provide half-width down arrow
but some Japanese fonts such as IPA Gothic and MS Gothic do full-width down arrow
from a kind of historical reason.
Post by Norman Diamond
Post by Norman Diamond
The version of Konsole that was included in Suse 11.0 did know that downarrow is
full-width.  I never had to look at what the default font was because it just
worked.
Konsole's character layouting was changed recently as far as I know
so that some (flexible-width) bold fonts can be placed correctly.
I guess it causes this problem.
Post by Norman Diamond
In the 固定幅フォントを選択 - Konsole panel, the bottom part is a text box.
I can type or paste text in the text box.
I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows,
and they display correctly in that text box.
Konsole corrupts the display in its main window, and the preview window does not preview the
corruption.
Unfortunately, the preview is for font selection dialog and Qt widgets.
It is not preview of Konsole.
Maybe, the preview should be removed.
Fuminobu TAKEYAMA
Post by Norman Diamond
There is a worse bug.
In the 固定幅フォントを選択 - Konsole panel, the bottom part is a text box.
I can type or paste text in the text box.
I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows,
and they display correctly in that text box.
Konsole corrupts the display in its main window, and the preview window does not preview the corruption.
Yours sincerely,Norman Diamond
Post by Norman Diamond
I click 設定 > 現在のプロファイルを編集(E)... > 外観
In フォント > プレビュー it says Monospace
Oh now I see, next to フォントサイズ where it lets me change the size, there's a
clickable button to select a font.
In that panel it says モノスペース instead of Monospace.
Thank you for the workaround.  I will experiment with some fonts.  I would like
to recommend that the preview box should include some full-width characters so
that users can judge more easily.  If a font doesn't include glyphs for
full-width characters then users will see immediately that they shouldn't
experiment with such a font.
I still think there's a bug.
Konsole knows that the glyph for 字 is full-width, so Konsole should know that
the glyph for downarrow is full-width.
The version of Konsole that was included in Suse 11.0 did know that downarrow is
full-width.  I never had to look at what the default font was because it just
worked.
Yours sincerely,
Norman Diamond
   Ok, Monospace is an alias of IPAGothic on your environment.
   The font, IPAGothic provides double-width down-allow glyph
   although Konsole assumes that the glyph should have the same width as
   glyphs such as 'a' and '0'.
   A solution is to use another font such as DejaVu Sans Mono.
   Settings > Edit Current Profile... > Appearance
   If you do not need Japanese fonts, please uninstall ipa-gothic-fonts.
   So another font is to be selected for Monospace.
   Fuminobu TAKEYAMA
    Yes the font is Monospace.  I did not change it from the default.  A
   Konsole settings panel displayed the name Monospace and allowed me to
change the
   font size but didn't allow me to select a different font.
    For this e-mail I use Yahoo's web interface from Internet Explorer
   under Windows, so I translated the output of fc-match -v and fc-match -v
   Monospace from UTF-8 to Microsoft's Unicode encoding.  Yahoo's web
   interface will scramble the text that I will paste below, so I ALSO attach
the
   outputs as files.
    Output of fc-match -v
    Pattern has 41 elts (size 48)
      family: "IPA Pゴシック"(s) "IPAPGothic"(s)
      familylang: "ja"(s) "en"(s)
      style: "Regular"(w)
      stylelang: "en"(w)
      fullname: "IPAPGothic"(w) "IPA Pゴシック"(w)
      fullnamelang: "en"(w) "ja"(w)
      slant: 0(i)(s)
      weight: 80(i)(s)
      width: 100(i)(s)
      size: 12(f)(s)
      pixelsize: 12.5(f)(s)
      foundry: "unknown"(w)
      antialias: True(w)
      hintstyle: 0(i)(w)
      hinting: True(w)
      verticallayout: False(s)
      autohint: True(w)
      globaladvance: True(s)
      file: "/usr/share/fonts/truetype/ipagp.ttf"(w)
      index: 0(i)(w)
      outline: True(w)
      scalable: True(w)
      dpi: 75(f)(s)
      rgba: 5(i)(w)
      scale: 1(f)(s)
      [snip]
    (w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
      fontversion: 198574(i)(s)
      capability: "otlayout:hani otlayout:kana otlayout:latn"(w)
      fontformat: "TrueType"(w)
      embeddedbitmap: True(w)
      decorative: False(s)
      lcdfilter: 0(i)(w)
      namelang: "ja"(s)
      prgname: "fc-match"(s)
"sha256:a63f6153841e56ec9df1b31a54d98d3f41d8aaa1c3d8df30a33502296f5b8068"(w)
      postscriptname: "IPAPGothic"(w)
      force_hintstyle: "none"(w)
      force_autohint: False(w)
      force_bw: False(w)
      force_bw_monospace: False(w)
    Output of fc-match -v Monospace
    Pattern has 42 elts (size 48)
      family: "IPAゴシック"(s) "IPAGothic"(s)
      familylang: "ja"(s) "en"(s)
      style: "Regular"(w)
      stylelang: "en"(w)
      fullname: "IPAGothic"(w) "IPAゴシック"(w)
      fullnamelang: "en"(w) "ja"(w)
      slant: 0(i)(s)
      weight: 80(i)(s)
      width: 100(i)(s)
      size: 12(f)(s)
      pixelsize: 12.5(f)(s)
      spacing: 90(i)(w)
      foundry: "unknown"(w)
      antialias: True(w)
      hintstyle: 0(i)(w)
      hinting: True(w)
      verticallayout: False(s)
      autohint: True(w)
      globaladvance: True(s)
      file: "/usr/share/fonts/truetype/ipag.ttf"(w)
      index: 0(i)(w)
      outline: True(w)
      scalable: True(w)
      dpi: 75(f)(s)
      rgba: 5(i)(w)
      scale: 1(f)(s)
      [snip]
    (w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
      fontversion: 198574(i)(s)
      capability: "otlayout:hani otlayout:kana otlayout:latn"(w)
      fontformat: "TrueType"(w)
      embeddedbitmap: False(w)
      decorative: False(s)
      lcdfilter: 0(i)(w)
      namelang: "ja"(s)
      prgname: "fc-match"(s)
"sha256:503af4a8b84d1079b8e2e358dc7f7a7fb8cb7a1f212f35eaef6782dbfc75a55e"(w)
      postscriptname: "IPAGothic"(w)
      force_hintstyle: "none"(w)
      force_autohint: False(w)
      force_bw: False(w)
      force_bw_monospace: False(w)
    ----- Original Message -----
      Date: 2015/6/10, Wed 19:43
      Subject: Re: [opensuse-m17n] Konsole displays some
characters
   wrong
      Hi,
      Which font do you use on Konsole?
      Then, can you share the result of fc-match -v?
      If the font is "Monospace" then run the following
      $ fc-match -v Monospace
      # you can snip off "charset:" section
      Fuminobu TAKEYAMA
       In openSUSE 13.2, in Konsole, downarrow↓displays↓wrong.
       The file encoding, Konsole terminal's encoding, and
   vim's
    encoding
      are all UTF-8.
       Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three.
       vim counts characters properly but the display is
wrong.
       more, cat, etc., display wrong.
       I used Suse 11.0 for six years.  Konsole had no problem
with
    downarrows.
       Yours sincerely,
       Norman Diamond
      --
   --
--
--
  
--
To unsubscribe, e-mail: opensuse-m17n+***@opensuse.org
To contact the owner, e-mail: opensuse-m17n+***@opensuse.org
Fuminobu TAKEYAMA
2015-06-13 02:26:15 UTC
Permalink
I do not agree with your statement that downarrow should be displayed as half-width. It is still
the same character that it used to be in ShiftJIS and EUC encodings, and the adoption of Unicode
and UTF-8 does not change the width of the glyph.
The ambiguous width problem have been discussed for a long time.
There is no way to distinguish CJK down arrow and non-CJK down arrow.
If I the character becomes half-width when I edit source files in Konsole, but it's still
full-width when other people edit the same source files in a Gnome terminal or even in Windows,
it will result in people scrambling source files.
If they edit GNOME terminal without CJK fonts, arrows are half-width.
Even if we resolve this Konsole bug, your problem will not be resolved.

You still need to keep in mind that down arrow (and some other characters) have
ambiguous width.
I do not know who decided that downarrow should have an ambiguous width.
If the Unicode consortium decided that, they're causing problems.
They did:
http://www.unicode.org/reports/tr11/tr11-28.html#Ambiguous
http://ja.wikipedia.org/wiki/東アジアの文字幅


Anyway, let's fix this Konsole's bug.

It is quite difficult to propose a new Unicode specification that can be
acceptable for both CJK and non-CJK people.

Fuminobu TAKEYAMA
Since the font selection dialog's preview uses Qt widgets instead of whatever Konsole uses, we
know that the full-width downarrow character is displayed as full-width by these Linux tools.
Furthermore in Suse 11.0 Konsole itself also displayed the full-width downarrow character as
full-width. So I think this bug is in the newer version of Konsole itself, not in the rest of
Linux. I can't say if the preview window should be removed because it doesn't really provide a
preview, but it helped narrow down this bug.
I do not agree with your statement that downarrow should be displayed as half-width. It is still
the same character that it used to be in ShiftJIS and EUC encodings, and the adoption of Unicode
and UTF-8 does not change the width of the glyph.
If I the character becomes half-width when I edit source files in Konsole, but it's still
full-width when other people edit the same source files in a Gnome terminal or even in Windows,
it will result in people scrambling source files.
Full-width a and half-width a do not have ambiguous widths. Full-width ア and half-width ア do not
have ambiguous widths. I do not know who decided that downarrow should have an ambiguous width.
If the Unicode consortium decided that, they're causing problems.
Yours sincerely, Norman Diamond
----- Forwarded Message -----
01:30 Subject: Re: [opensuse-m17n] Konsole displays some characters wrong
Post by Norman Diamond
I still think there's a bug.
Yes, this is clearly a Japanese specific bug. Could you file a bug report for openSUSE and
KDE?
I saw a patch to fix the same problem with emacs about 5 years ago. So I think this problem can
be resolved by some hacks.
Post by Norman Diamond
Konsole knows that the glyph for 字 is full-width, so Konsole should know that the glyph for
downarrow is full-width.
Down arrow is known as ambiguous-width character. Monospaced font should provide half-width
down arrow but some Japanese fonts such as IPA Gothic and MS Gothic do full-width down arrow
from a kind of historical reason.
Post by Norman Diamond
The version of Konsole that was included in Suse 11.0 did know that downarrow is
full-width. I never had to look at what the default font was because it just worked.
Konsole's character layouting was changed recently as far as I know so that some
(flexible-width) bold fonts can be placed correctly.
I guess it causes this problem.
In the 固定幅フォントを選択 - Konsole panel, the bottom part is a text box. I can type or paste text in
the text box. I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they display correctly in that
text box. Konsole corrupts the display in its main window, and the preview window does not
preview the
corruption.
Unfortunately, the preview is for font selection dialog and Qt widgets. It is not preview of
Konsole.
Maybe, the preview should be removed.
Fuminobu TAKEYAMA
There is a worse bug.
In the 固定幅フォントを選択 - Konsole panel, the bottom part is a text box. I can type or paste text in
the text box. I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they display correctly in that
text box. Konsole corrupts the display in its main window, and the preview window does not
preview the corruption.
Yours sincerely,Norman Diamond
Post by Norman Diamond
I click 設定 > 現在のプロファイルを編集(E)... > 外観 In フォント > プレビュー it says Monospace Oh now I see, next
to フォントサイズ where it lets me change the size, there's a clickable button to select a font.
In that panel it says モノスペース instead of Monospace.
Thank you for the workaround. I will experiment with some fonts. I would like to
recommend that the preview box should include some full-width characters so that users can
judge more easily. If a font doesn't include glyphs for full-width characters then users
will see immediately that they shouldn't experiment with such a font.
I still think there's a bug.
Konsole knows that the glyph for 字 is full-width, so Konsole should know that the glyph for
downarrow is full-width.
The version of Konsole that was included in Suse 11.0 did know that downarrow is
full-width. I never had to look at what the default font was because it just worked.
Yours sincerely, Norman Diamond
Post by Fuminobu TAKEYAMA
Ok, Monospace is an alias of IPAGothic on your environment.
The font, IPAGothic provides double-width down-allow glyph although Konsole assumes that
the glyph should have the same width as glyphs such as 'a' and '0'.
Settings > Edit Current Profile... > Appearance
If you do not need Japanese fonts, please uninstall ipa-gothic-fonts. So another font is
to be selected for Monospace.
Fuminobu TAKEYAMA
Yes the font is Monospace. I did not change it from the default. A
Konsole settings panel displayed the name Monospace and allowed me to
change the
Post by Fuminobu TAKEYAMA
font size but didn't allow me to select a different font.
For this e-mail I use Yahoo's web interface from Internet Explorer
under Windows, so I translated the output of fc-match -v and fc-match -v Monospace from
UTF-8 to Microsoft's Unicode encoding. Yahoo's web interface will scramble the text that
I will paste below, so I ALSO attach
the
Post by Fuminobu TAKEYAMA
outputs as files.
Output of fc-match -v
"ja"(s) "en"(s) style: "Regular"(w) stylelang: "en"(w) fullname: "IPAPGothic"(w) "IPA P
True(w) hintstyle: 0(i)(w) hinting: True(w) verticallayout: False(s) autohint: True(w)
globaladvance: True(s) file: "/usr/share/fonts/truetype/ipagp.ttf"(w) index: 0(i)(w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
fontversion: 198574(i)(s)
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
capability: "otlayout:hani otlayout:kana otlayout:latn"(w) fontformat: "TrueType"(w)
embeddedbitmap: True(w) decorative: False(s) lcdfilter: 0(i)(w) namelang: "ja"(s)
"sha256:a63f6153841e56ec9df1b31a54d98d3f41d8aaa1c3d8df30a33502296f5b8068"(w)
Post by Fuminobu TAKEYAMA
postscriptname: "IPAPGothic"(w) force_hintstyle: "none"(w) force_autohint: False(w)
force_bw: False(w) force_bw_monospace: False(w)
Output of fc-match -v Monospace
Pattern has 42 elts (size 48) family: "IPAゴシック"(s) "IPAGothic"(s) familylang: "ja"(s)
"en"(s) style: "Regular"(w) stylelang: "en"(w) fullname: "IPAGothic"(w) "IPAゴシック"(w)
True(w) hintstyle: 0(i)(w) hinting: True(w) verticallayout: False(s) autohint: True(w)
globaladvance: True(s) file: "/usr/share/fonts/truetype/ipag.ttf"(w) index: 0(i)(w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
fontversion: 198574(i)(s)
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
capability: "otlayout:hani otlayout:kana otlayout:latn"(w) fontformat: "TrueType"(w)
embeddedbitmap: False(w) decorative: False(s) lcdfilter: 0(i)(w) namelang: "ja"(s)
"sha256:503af4a8b84d1079b8e2e358dc7f7a7fb8cb7a1f212f35eaef6782dbfc75a55e"(w)
Post by Fuminobu TAKEYAMA
postscriptname: "IPAGothic"(w) force_hintstyle: "none"(w) force_autohint: False(w)
force_bw: False(w) force_bw_monospace: False(w)
----- Original Message -----
2015/6/10, Wed 19:43 Subject: Re: [opensuse-m17n] Konsole displays some
characters
Post by Fuminobu TAKEYAMA
wrong
Hi,
Which font do you use on Konsole?
Then, can you share the result of fc-match -v? If the font is "Monospace" then run
the following
$ fc-match -v Monospace
# you can snip off "charset:" section
Fuminobu TAKEYAMA
In openSUSE 13.2, in Konsole, downarrow↓displays↓wrong. The file encoding,
Konsole terminal's encoding, and
vim's
encoding
are all UTF-8.
Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three. vim counts characters properly
but the display is
wrong.
Post by Fuminobu TAKEYAMA
more, cat, etc., display wrong.
I used Suse 11.0 for six years. Konsole had no problem
with
Post by Fuminobu TAKEYAMA
downarrows.
Yours sincerely, Norman Diamond
--
To unsubscribe, e-mail: opensuse-m17n+***@opensuse.org
To contact the owner, e-mail: opensuse-m17n+***@opensuse.org
Norman Diamond
2015-06-13 02:50:37 UTC
Permalink
I think the Unicode consortium should be taught that full-width and half-width downarrow can be distinguished just as full-width Romaji can be distinguished from half-width Romaji, without even needing CJK.  Even a full-width plain blank space is distinguished from a half-width plain blank space.

However, anyone who edits these source files will likely have CJK fonts.  The comments are mostly in Japanese.  Strings have mostly been moved to data files, but changes to the code are generally accompanied by adding or editing strings that are also Japanese.
Yours sincerely,
Norman Diamond
Post by Norman Diamond
Post by Norman Diamond
I do not agree with your statement that downarrow should be displayed as
half-width.  It is still
Post by Norman Diamond
the same character that it used to be in ShiftJIS and EUC encodings, and
the adoption of Unicode
Post by Norman Diamond
and UTF-8 does not change the width of the glyph.
The ambiguous width problem have been discussed for a long time.
There is no way to distinguish CJK down arrow and non-CJK down arrow.
Post by Norman Diamond
If I the character becomes half-width when I edit source files in Konsole,
but it's still
Post by Norman Diamond
full-width when other people edit the same source files in a Gnome terminal
or even in Windows,
Post by Norman Diamond
it will result in people scrambling source files.
If they edit GNOME terminal without CJK fonts, arrows are half-width.
Even if we resolve this Konsole bug, your problem will not be resolved.
You still need to keep in mind that down arrow (and some other characters) have
ambiguous width.
Post by Norman Diamond
I do not know who decided that downarrow should have an ambiguous width.
If the Unicode consortium decided that, they're causing problems.
http://www.unicode.org/reports/tr11/tr11-28.html#Ambiguous
http://ja.wikipedia.org/wiki/東アジアの文字幅
Anyway, let's fix this Konsole's bug.
It is quite difficult to propose a new Unicode specification that can be
acceptable for both CJK and non-CJK people.
Fuminobu TAKEYAMA
Post by Norman Diamond
Since the font selection dialog's preview uses Qt widgets instead of
whatever Konsole uses, we
Post by Norman Diamond
know that the full-width downarrow character is displayed as full-width by
these Linux tools.
Post by Norman Diamond
Furthermore in Suse 11.0 Konsole itself also displayed the full-width
downarrow character as
Post by Norman Diamond
full-width.  So I think this bug is in the newer version of Konsole itself,
not in the rest of
Post by Norman Diamond
Linux.  I can't say if the preview window should be removed because it
doesn't really provide a
Post by Norman Diamond
preview, but it helped narrow down this bug.
I do not agree with your statement that downarrow should be displayed as
half-width.  It is still
Post by Norman Diamond
the same character that it used to be in ShiftJIS and EUC encodings, and
the adoption of Unicode
Post by Norman Diamond
and UTF-8 does not change the width of the glyph.
If I the character becomes half-width when I edit source files in Konsole,
but it's still
Post by Norman Diamond
full-width when other people edit the same source files in a Gnome terminal
or even in Windows,
Post by Norman Diamond
it will result in people scrambling source files.
Full-width a and half-width a do not have ambiguous widths.  Full-width ア
and half-width ア do not
Post by Norman Diamond
have ambiguous widths.  I do not know who decided that downarrow should
have an ambiguous width.
Post by Norman Diamond
If the Unicode consortium decided that, they're causing problems.
Yours sincerely, Norman Diamond
----- Forwarded Message -----
01:30 Subject: Re: [opensuse-m17n] Konsole displays some characters
wrong
Post by Norman Diamond
Post by Norman Diamond
I still think there's a bug.
Yes, this is clearly a Japanese specific bug. Could you file a bug
report for openSUSE and
Post by Norman Diamond
KDE?
I saw a patch to fix the same problem with emacs about 5 years ago. So
I think this problem can
Post by Norman Diamond
be resolved by some hacks.
Post by Norman Diamond
Konsole knows that the glyph for 字 is full-width, so Konsole
should know that the glyph for
Post by Norman Diamond
Post by Norman Diamond
downarrow is full-width.
Down arrow is known as ambiguous-width character. Monospaced font
should provide half-width
Post by Norman Diamond
down arrow but some Japanese fonts such as IPA Gothic and MS Gothic do
full-width down arrow
Post by Norman Diamond
from a kind of historical reason.
Post by Norman Diamond
The version of Konsole that was included in Suse 11.0 did know
that downarrow is
Post by Norman Diamond
Post by Norman Diamond
full-width.  I never had to look at what the default font was
because it just worked.
Post by Norman Diamond
Konsole's character layouting was changed recently as far as I know
so that some
Post by Norman Diamond
(flexible-width) bold fonts can be placed correctly.
I guess it causes this problem.
In the 固定幅フォントを選択 - Konsole panel, the bottom part is a text box. I
can type or paste text in
Post by Norman Diamond
the text box. I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they
display correctly in that
Post by Norman Diamond
text box. Konsole corrupts the display in its main window, and the
preview window does not
Post by Norman Diamond
preview the
corruption.
Unfortunately, the preview is for font selection dialog and Qt widgets.
It is not preview of
Post by Norman Diamond
Konsole.
Maybe, the preview should be removed.
Fuminobu TAKEYAMA
There is a worse bug.
In the 固定幅フォントを選択 - Konsole panel, the bottom part is a text box. I
can type or paste text in
Post by Norman Diamond
the text box. I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they
display correctly in that
Post by Norman Diamond
text box. Konsole corrupts the display in its main window, and the
preview window does not
Post by Norman Diamond
preview the corruption.
Yours sincerely,Norman Diamond
Post by Norman Diamond
I click 設定 > 現在のプロファイルを編集(E)... > 外観 In フォント > プレビュー
it says Monospace Oh now I see, next
Post by Norman Diamond
Post by Norman Diamond
to フォントサイズ where it lets me change the size, there's a
clickable button to select a font.
Post by Norman Diamond
Post by Norman Diamond
In that panel it says モノスペース instead of Monospace.
Thank you for the workaround.  I will experiment with some
fonts.  I would like to
Post by Norman Diamond
Post by Norman Diamond
recommend that the preview box should include some full-width
characters so that users can
Post by Norman Diamond
Post by Norman Diamond
judge more easily.  If a font doesn't include glyphs for
full-width characters then users
Post by Norman Diamond
Post by Norman Diamond
will see immediately that they shouldn't experiment with
such a font.
Post by Norman Diamond
Post by Norman Diamond
I still think there's a bug.
Konsole knows that the glyph for 字 is full-width, so Konsole
should know that the glyph for
Post by Norman Diamond
Post by Norman Diamond
downarrow is full-width.
The version of Konsole that was included in Suse 11.0 did know
that downarrow is
Post by Norman Diamond
Post by Norman Diamond
full-width.  I never had to look at what the default font was
because it just worked.
Post by Norman Diamond
Post by Norman Diamond
Yours sincerely, Norman Diamond
Post by Fuminobu TAKEYAMA
Ok, Monospace is an alias of IPAGothic on your environment.
The font, IPAGothic provides double-width down-allow glyph
although Konsole assumes that
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
the glyph should have the same width as glyphs such as
'a' and '0'.
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
A solution is to use another font such as DejaVu Sans Mono.
Settings > Edit Current Profile... > Appearance
If you do not need Japanese fonts, please uninstall
ipa-gothic-fonts. So another font is
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
to be selected for Monospace.
Fuminobu TAKEYAMA
Yes the font is Monospace.  I did not change it from
the default.  A
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
Konsole settings panel displayed the name Monospace and
allowed me to
Post by Norman Diamond
Post by Norman Diamond
change the
Post by Fuminobu TAKEYAMA
font size but didn't allow me to select a different
font.
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
For this e-mail I use Yahoo's web interface from
Internet Explorer
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
under Windows, so I translated the output of fc-match -v
and fc-match -v Monospace from
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
UTF-8 to Microsoft's Unicode encoding.  Yahoo's web
interface will scramble the text that
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
I will paste below, so I ALSO attach
the
Post by Fuminobu TAKEYAMA
outputs as files.
Output of fc-match -v
Pattern has 41 elts (size 48) family: "IPA
"IPAPGothic"(w) "IPA P
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
ゴシック"(w) fullnamelang: "en"(w)
True(w) hintstyle: 0(i)(w) hinting: True(w)
verticallayout: False(s) autohint: True(w)
"/usr/share/fonts/truetype/ipagp.ttf"(w) index: 0(i)(w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
fontversion: 198574(i)(s)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
capability: "otlayout:hani otlayout:kana
otlayout:latn"(w) fontformat: "TrueType"(w)
0(i)(w) namelang: "ja"(s)
"sha256:a63f6153841e56ec9df1b31a54d98d3f41d8aaa1c3d8df30a33502296f5b8068"(w)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
postscriptname: "IPAPGothic"(w)
force_hintstyle: "none"(w) force_autohint: False(w)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
force_bw: False(w) force_bw_monospace: False(w)
Output of fc-match -v Monospace
"IPAゴシック"(s) "IPAGothic"(s) familylang: "ja"(s)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
"en"(s) style: "Regular"(w)
stylelang: "en"(w) fullname: "IPAGothic"(w)
"IPAゴシック"(w)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
fullnamelang: "en"(w) "ja"(w)
12(f)(s) pixelsize: 12.5(f)(s) spacing: 90(i)(w)
True(w) hintstyle: 0(i)(w) hinting: True(w)
verticallayout: False(s) autohint: True(w)
"/usr/share/fonts/truetype/ipag.ttf"(w) index: 0(i)(w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
fontversion: 198574(i)(s)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
capability: "otlayout:hani otlayout:kana
otlayout:latn"(w) fontformat: "TrueType"(w)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
embeddedbitmap: False(w) decorative: False(s)
lcdfilter: 0(i)(w) namelang: "ja"(s)
"sha256:503af4a8b84d1079b8e2e358dc7f7a7fb8cb7a1f212f35eaef6782dbfc75a55e"(w)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
postscriptname: "IPAGothic"(w)
force_hintstyle: "none"(w) force_autohint: False(w)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
force_bw: False(w) force_bw_monospace: False(w)
----- Original Message -----
[opensuse-m17n] Konsole displays some
Post by Norman Diamond
Post by Norman Diamond
characters
Post by Fuminobu TAKEYAMA
wrong
Post by Fuminobu TAKEYAMA
Hi,
Which font do you use on Konsole?
Then, can you share the result of fc-match -v?
If the font is "Monospace" then run
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
Post by Fuminobu TAKEYAMA
the following
$ fc-match -v Monospace
# you can snip off "charset:" section
Fuminobu TAKEYAMA
Post by Norman Diamond
In openSUSE 13.2, in Konsole,
downarrow↓displays↓wrong. The file encoding,
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
Post by Fuminobu TAKEYAMA
Post by Norman Diamond
Konsole terminal's encoding, and
vim's
encoding
Post by Fuminobu TAKEYAMA
are all UTF-8.
Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three. vim counts characters properly
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
Post by Fuminobu TAKEYAMA
Post by Norman Diamond
but the display is
wrong.
Post by Fuminobu TAKEYAMA
Post by Fuminobu TAKEYAMA
Post by Norman Diamond
more, cat, etc., display wrong.
I used Suse 11.0 for six years.  Konsole
had no problem
Post by Norman Diamond
Post by Norman Diamond
with
Post by Fuminobu TAKEYAMA
downarrows.
Post by Fuminobu TAKEYAMA
Post by Norman Diamond
Yours sincerely, Norman Diamond
--
--
To unsubscribe, e-mail: opensuse-m17n+***@opensuse.org
To contact the owner, e-mail: opensuse-m17n+***@opensuse.org
Takashi Iwai
2015-06-13 07:26:00 UTC
Permalink
At Sat, 13 Jun 2015 11:50:37 +0900 (JST),
Post by Norman Diamond
I think the Unicode consortium should be taught that full-width and half-width downarrow can be distinguished just as full-width Romaji can be distinguished from half-width Romaji, without even needing CJK.  Even a full-width plain blank space is distinguished from a half-width plain blank space.
It's 20 years too late :)
Post by Norman Diamond
However, anyone who edits these source files will likely have CJK fonts.  The comments are mostly in Japanese.  Strings have mostly been moved to data files, but changes to the code are generally accompanied by adding or editing strings that are also Japanese.
There is globaladvance attribute in the fontconfig and we used to set
it false for CJK for some time. The attribute itself is marked as
deprecated nowadays, but this might still work. You need to give it a
try.


Takashi
Post by Norman Diamond
Yours sincerely,
Norman Diamond
Post by Norman Diamond
Post by Norman Diamond
I do not agree with your statement that downarrow should be displayed as
half-width.  It is still
Post by Norman Diamond
the same character that it used to be in ShiftJIS and EUC encodings, and
the adoption of Unicode
Post by Norman Diamond
and UTF-8 does not change the width of the glyph.
The ambiguous width problem have been discussed for a long time.
There is no way to distinguish CJK down arrow and non-CJK down arrow.
Post by Norman Diamond
If I the character becomes half-width when I edit source files in Konsole,
but it's still
Post by Norman Diamond
full-width when other people edit the same source files in a Gnome terminal
or even in Windows,
Post by Norman Diamond
it will result in people scrambling source files.
If they edit GNOME terminal without CJK fonts, arrows are half-width.
Even if we resolve this Konsole bug, your problem will not be resolved.
You still need to keep in mind that down arrow (and some other characters) have
ambiguous width.
Post by Norman Diamond
I do not know who decided that downarrow should have an ambiguous width.
If the Unicode consortium decided that, they're causing problems.
http://www.unicode.org/reports/tr11/tr11-28.html#Ambiguous
http://ja.wikipedia.org/wiki/東アジアの文字幅
Anyway, let's fix this Konsole's bug.
It is quite difficult to propose a new Unicode specification that can be
acceptable for both CJK and non-CJK people.
Fuminobu TAKEYAMA
Post by Norman Diamond
Since the font selection dialog's preview uses Qt widgets instead of
whatever Konsole uses, we
Post by Norman Diamond
know that the full-width downarrow character is displayed as full-width by
these Linux tools.
Post by Norman Diamond
Furthermore in Suse 11.0 Konsole itself also displayed the full-width
downarrow character as
Post by Norman Diamond
full-width.  So I think this bug is in the newer version of Konsole itself,
not in the rest of
Post by Norman Diamond
Linux.  I can't say if the preview window should be removed because it
doesn't really provide a
Post by Norman Diamond
preview, but it helped narrow down this bug.
I do not agree with your statement that downarrow should be displayed as
half-width.  It is still
Post by Norman Diamond
the same character that it used to be in ShiftJIS and EUC encodings, and
the adoption of Unicode
Post by Norman Diamond
and UTF-8 does not change the width of the glyph.
If I the character becomes half-width when I edit source files in Konsole,
but it's still
Post by Norman Diamond
full-width when other people edit the same source files in a Gnome terminal
or even in Windows,
Post by Norman Diamond
it will result in people scrambling source files.
Full-width a and half-width a do not have ambiguous widths.  Full-width ア
and half-width ア do not
Post by Norman Diamond
have ambiguous widths.  I do not know who decided that downarrow should
have an ambiguous width.
Post by Norman Diamond
If the Unicode consortium decided that, they're causing problems.
Yours sincerely, Norman Diamond
----- Forwarded Message -----
01:30 Subject: Re: [opensuse-m17n] Konsole displays some characters
wrong
Post by Norman Diamond
Post by Norman Diamond
I still think there's a bug.
Yes, this is clearly a Japanese specific bug. Could you file a bug
report for openSUSE and
Post by Norman Diamond
KDE?
I saw a patch to fix the same problem with emacs about 5 years ago. So
I think this problem can
Post by Norman Diamond
be resolved by some hacks.
Post by Norman Diamond
Konsole knows that the glyph for 字 is full-width, so Konsole
should know that the glyph for
Post by Norman Diamond
Post by Norman Diamond
downarrow is full-width.
Down arrow is known as ambiguous-width character. Monospaced font
should provide half-width
Post by Norman Diamond
down arrow but some Japanese fonts such as IPA Gothic and MS Gothic do
full-width down arrow
Post by Norman Diamond
from a kind of historical reason.
Post by Norman Diamond
The version of Konsole that was included in Suse 11.0 did know
that downarrow is
Post by Norman Diamond
Post by Norman Diamond
full-width.  I never had to look at what the default font was
because it just worked.
Post by Norman Diamond
Konsole's character layouting was changed recently as far as I know
so that some
Post by Norman Diamond
(flexible-width) bold fonts can be placed correctly.
I guess it causes this problem.
In the 固定幅フォントを選択 - Konsole panel, the bottom part is a text box. I
can type or paste text in
Post by Norman Diamond
the text box. I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they
display correctly in that
Post by Norman Diamond
text box. Konsole corrupts the display in its main window, and the
preview window does not
Post by Norman Diamond
preview the
corruption.
Unfortunately, the preview is for font selection dialog and Qt widgets.
It is not preview of
Post by Norman Diamond
Konsole.
Maybe, the preview should be removed.
Fuminobu TAKEYAMA
There is a worse bug.
In the 固定幅フォントを選択 - Konsole panel, the bottom part is a text box. I
can type or paste text in
Post by Norman Diamond
the text box. I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they
display correctly in that
Post by Norman Diamond
text box. Konsole corrupts the display in its main window, and the
preview window does not
Post by Norman Diamond
preview the corruption.
Yours sincerely,Norman Diamond
Post by Norman Diamond
I click 設定 > 現在のプロファイルを編集(E)... > 外観 In フォント > プレビュー
it says Monospace Oh now I see, next
Post by Norman Diamond
Post by Norman Diamond
to フォントサイズ where it lets me change the size, there's a
clickable button to select a font.
Post by Norman Diamond
Post by Norman Diamond
In that panel it says モノスペース instead of Monospace.
Thank you for the workaround.  I will experiment with some
fonts.  I would like to
Post by Norman Diamond
Post by Norman Diamond
recommend that the preview box should include some full-width
characters so that users can
Post by Norman Diamond
Post by Norman Diamond
judge more easily.  If a font doesn't include glyphs for
full-width characters then users
Post by Norman Diamond
Post by Norman Diamond
will see immediately that they shouldn't experiment with
such a font.
Post by Norman Diamond
Post by Norman Diamond
I still think there's a bug.
Konsole knows that the glyph for 字 is full-width, so Konsole
should know that the glyph for
Post by Norman Diamond
Post by Norman Diamond
downarrow is full-width.
The version of Konsole that was included in Suse 11.0 did know
that downarrow is
Post by Norman Diamond
Post by Norman Diamond
full-width.  I never had to look at what the default font was
because it just worked.
Post by Norman Diamond
Post by Norman Diamond
Yours sincerely, Norman Diamond
Post by Fuminobu TAKEYAMA
Ok, Monospace is an alias of IPAGothic on your environment.
The font, IPAGothic provides double-width down-allow glyph
although Konsole assumes that
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
the glyph should have the same width as glyphs such as
'a' and '0'.
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
A solution is to use another font such as DejaVu Sans Mono.
Settings > Edit Current Profile... > Appearance
If you do not need Japanese fonts, please uninstall
ipa-gothic-fonts. So another font is
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
to be selected for Monospace.
Fuminobu TAKEYAMA
Yes the font is Monospace.  I did not change it from
the default.  A
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
Konsole settings panel displayed the name Monospace and
allowed me to
Post by Norman Diamond
Post by Norman Diamond
change the
Post by Fuminobu TAKEYAMA
font size but didn't allow me to select a different
font.
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
For this e-mail I use Yahoo's web interface from
Internet Explorer
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
under Windows, so I translated the output of fc-match -v
and fc-match -v Monospace from
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
UTF-8 to Microsoft's Unicode encoding.  Yahoo's web
interface will scramble the text that
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
I will paste below, so I ALSO attach
the
Post by Fuminobu TAKEYAMA
outputs as files.
Output of fc-match -v
Pattern has 41 elts (size 48) family: "IPA
"IPAPGothic"(w) "IPA P
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
ゴシック"(w) fullnamelang: "en"(w)
True(w) hintstyle: 0(i)(w) hinting: True(w)
verticallayout: False(s) autohint: True(w)
"/usr/share/fonts/truetype/ipagp.ttf"(w) index: 0(i)(w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
fontversion: 198574(i)(s)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
capability: "otlayout:hani otlayout:kana
otlayout:latn"(w) fontformat: "TrueType"(w)
0(i)(w) namelang: "ja"(s)
"sha256:a63f6153841e56ec9df1b31a54d98d3f41d8aaa1c3d8df30a33502296f5b8068"(w)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
postscriptname: "IPAPGothic"(w)
force_hintstyle: "none"(w) force_autohint: False(w)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
force_bw: False(w) force_bw_monospace: False(w)
Output of fc-match -v Monospace
"IPAゴシック"(s) "IPAGothic"(s) familylang: "ja"(s)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
"en"(s) style: "Regular"(w)
stylelang: "en"(w) fullname: "IPAGothic"(w)
"IPAゴシック"(w)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
fullnamelang: "en"(w) "ja"(w)
12(f)(s) pixelsize: 12.5(f)(s) spacing: 90(i)(w)
True(w) hintstyle: 0(i)(w) hinting: True(w)
verticallayout: False(s) autohint: True(w)
"/usr/share/fonts/truetype/ipag.ttf"(w) index: 0(i)(w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
fontversion: 198574(i)(s)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
capability: "otlayout:hani otlayout:kana
otlayout:latn"(w) fontformat: "TrueType"(w)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
embeddedbitmap: False(w) decorative: False(s)
lcdfilter: 0(i)(w) namelang: "ja"(s)
"sha256:503af4a8b84d1079b8e2e358dc7f7a7fb8cb7a1f212f35eaef6782dbfc75a55e"(w)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
postscriptname: "IPAGothic"(w)
force_hintstyle: "none"(w) force_autohint: False(w)
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
force_bw: False(w) force_bw_monospace: False(w)
----- Original Message -----
[opensuse-m17n] Konsole displays some
Post by Norman Diamond
Post by Norman Diamond
characters
Post by Fuminobu TAKEYAMA
wrong
Post by Fuminobu TAKEYAMA
Hi,
Which font do you use on Konsole?
Then, can you share the result of fc-match -v?
If the font is "Monospace" then run
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
Post by Fuminobu TAKEYAMA
the following
$ fc-match -v Monospace
# you can snip off "charset:" section
Fuminobu TAKEYAMA
Post by Norman Diamond
In openSUSE 13.2, in Konsole,
downarrow↓displays↓wrong. The file encoding,
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
Post by Fuminobu TAKEYAMA
Post by Norman Diamond
Konsole terminal's encoding, and
vim's
encoding
Post by Fuminobu TAKEYAMA
are all UTF-8.
Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three. vim counts characters properly
Post by Norman Diamond
Post by Norman Diamond
Post by Fuminobu TAKEYAMA
Post by Fuminobu TAKEYAMA
Post by Norman Diamond
but the display is
wrong.
Post by Fuminobu TAKEYAMA
Post by Fuminobu TAKEYAMA
Post by Norman Diamond
more, cat, etc., display wrong.
I used Suse 11.0 for six years.  Konsole
had no problem
Post by Norman Diamond
Post by Norman Diamond
with
Post by Fuminobu TAKEYAMA
downarrows.
Post by Fuminobu TAKEYAMA
Post by Norman Diamond
Yours sincerely, Norman Diamond
--
--
--
To unsubscribe, e-mail: opensuse-m17n+***@opensuse.org
To contact the owner, e-mail: opensuse-m17n+***@opensuse.org
Norman Diamond
2015-06-13 07:56:57 UTC
Permalink
OK, it could be 20 years too late to prevent ambiguous-width Romaji, katakana, and space characters from entering Unicode.

However, the Unicode consortium continues to add characters.  It isn't too late to add some necessary characters.

A half-width downarrow can be added with the same width as half-width space, Romaji, and katakana.  A full-width downarrow can be added with same same width as full-width space, Romaji, and katakana.  Ambiguous-width space, Romaji, and katakana can be added to help unify the performance of applications when the text doesn't happen to include arrows.
Sincerely,
Norman Diamond
Post by Norman Diamond
Post by Norman Diamond
I think the Unicode consortium should be taught that full-width and
half-width downarrow can be distinguished just as full-width Romaji can be
distinguished from half-width Romaji, without even needing CJK.  Even a
full-width plain blank space is distinguished from a half-width plain blank
space.
It's 20 years too late :)
Post by Norman Diamond
However, anyone who edits these source files will likely have CJK fonts. 
The comments are mostly in Japanese.  Strings have mostly been moved to data
files, but changes to the code are generally accompanied by adding or editing
strings that are also Japanese.
There is globaladvance attribute in the fontconfig and we used to set
it false for CJK for some time.  The attribute itself is marked as
deprecated nowadays, but this might still work.  You need to give it a
try.
Takashi
Post by Norman Diamond
Yours sincerely,
Norman Diamond
Post by Norman Diamond
  I do not agree with your statement that downarrow should be
displayed as
Post by Norman Diamond
Post by Norman Diamond
half-width.  It is still
  the same character that it used to be in ShiftJIS and EUC
encodings, and
Post by Norman Diamond
Post by Norman Diamond
the adoption of Unicode
  and UTF-8 does not change the width of the glyph.
The ambiguous width problem have been discussed for a long time.
There is no way to distinguish CJK down arrow and non-CJK down arrow.
  If I the character becomes half-width when I edit source files in
Konsole,
Post by Norman Diamond
Post by Norman Diamond
but it's still
  full-width when other people edit the same source files in a
Gnome terminal
Post by Norman Diamond
Post by Norman Diamond
or even in Windows,
  it will result in people scrambling source files.
If they edit GNOME terminal without CJK fonts, arrows are half-width.
Even if we resolve this Konsole bug, your problem will not be
resolved.
Post by Norman Diamond
Post by Norman Diamond
You still need to keep in mind that down arrow (and some other
characters) have
Post by Norman Diamond
Post by Norman Diamond
ambiguous width.
  I do not know who decided that downarrow should have an ambiguous
width.
Post by Norman Diamond
Post by Norman Diamond
  If the Unicode consortium decided that, they're causing
problems.
Post by Norman Diamond
Post by Norman Diamond
http://www.unicode.org/reports/tr11/tr11-28.html#Ambiguous
http://ja.wikipedia.org/wiki/東アジアの文字幅
Anyway, let's fix this Konsole's bug.
It is quite difficult to propose a new Unicode specification that can
be
Post by Norman Diamond
Post by Norman Diamond
acceptable for both CJK and non-CJK people.
Fuminobu TAKEYAMA
  Since the font selection dialog's preview uses Qt widgets
instead of
Post by Norman Diamond
Post by Norman Diamond
whatever Konsole uses, we
  know that the full-width downarrow character is displayed as
full-width by
Post by Norman Diamond
Post by Norman Diamond
these Linux tools.
  Furthermore in Suse 11.0 Konsole itself also displayed the
full-width
Post by Norman Diamond
Post by Norman Diamond
downarrow character as
  full-width.  So I think this bug is in the newer version of
Konsole itself,
Post by Norman Diamond
Post by Norman Diamond
not in the rest of
  Linux.  I can't say if the preview window should be removed
because it
Post by Norman Diamond
Post by Norman Diamond
doesn't really provide a
  preview, but it helped narrow down this bug.
  I do not agree with your statement that downarrow should be
displayed as
Post by Norman Diamond
Post by Norman Diamond
half-width.  It is still
  the same character that it used to be in ShiftJIS and EUC
encodings, and
Post by Norman Diamond
Post by Norman Diamond
the adoption of Unicode
  and UTF-8 does not change the width of the glyph.
  If I the character becomes half-width when I edit source files in
Konsole,
Post by Norman Diamond
Post by Norman Diamond
but it's still
  full-width when other people edit the same source files in a
Gnome terminal
Post by Norman Diamond
Post by Norman Diamond
or even in Windows,
  it will result in people scrambling source files.
  Full-width a and half-width a do not have ambiguous widths. 
Full-width ア
Post by Norman Diamond
Post by Norman Diamond
and half-width ア do not
  have ambiguous widths.  I do not know who decided that downarrow
should
Post by Norman Diamond
Post by Norman Diamond
have an ambiguous width.
  If the Unicode consortium decided that, they're causing
problems.
Post by Norman Diamond
Post by Norman Diamond
  Yours sincerely, Norman Diamond
  ----- Forwarded Message -----
  01:30 Subject: Re: [opensuse-m17n] Konsole displays some
characters
Post by Norman Diamond
Post by Norman Diamond
wrong
  I still think there's a bug.
  Yes, this is clearly a Japanese specific bug. Could you file
a bug
Post by Norman Diamond
Post by Norman Diamond
report for openSUSE and
  KDE?
  I saw a patch to fix the same problem with emacs about 5
years ago. So
Post by Norman Diamond
Post by Norman Diamond
I think this problem can
  be resolved by some hacks.
  Konsole knows that the glyph for 字 is full-width, so
Konsole
Post by Norman Diamond
Post by Norman Diamond
should know that the glyph for
  downarrow is full-width.
  Down arrow is known as ambiguous-width character. Monospaced
font
Post by Norman Diamond
Post by Norman Diamond
should provide half-width
  down arrow but some Japanese fonts such as IPA Gothic and MS
Gothic do
Post by Norman Diamond
Post by Norman Diamond
full-width down arrow
  from a kind of historical reason.
  The version of Konsole that was included in Suse 11.0
did know
Post by Norman Diamond
Post by Norman Diamond
that downarrow is
  full-width.  I never had to look at what the default
font was
Post by Norman Diamond
Post by Norman Diamond
because it just worked.
  Konsole's character layouting was changed recently as far
as I know
Post by Norman Diamond
Post by Norman Diamond
so that some
  (flexible-width) bold fonts can be placed correctly.
  I guess it causes this problem.
  In the 固定幅フォントを選択 - Konsole panel, the bottom part is a
text box. I
Post by Norman Diamond
Post by Norman Diamond
can type or paste text in
  the text box.
I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they
Post by Norman Diamond
Post by Norman Diamond
display correctly in that
  text box. Konsole corrupts the display in its main
window, and the
Post by Norman Diamond
Post by Norman Diamond
preview window does not
  preview the
  corruption.
  Unfortunately, the preview is for font selection dialog and
Qt widgets.
Post by Norman Diamond
Post by Norman Diamond
It is not preview of
  Konsole.
  Maybe, the preview should be removed.
  Fuminobu TAKEYAMA
  There is a worse bug.
  In the 固定幅フォントを選択 - Konsole panel, the bottom part is a
text box. I
Post by Norman Diamond
Post by Norman Diamond
can type or paste text in
  the text box.
I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they
Post by Norman Diamond
Post by Norman Diamond
display correctly in that
  text box. Konsole corrupts the display in its main
window, and the
Post by Norman Diamond
Post by Norman Diamond
preview window does not
  preview the corruption.
  Yours sincerely,Norman Diamond
  I click 設定 > 現在のプロファイルを編集(E)... > 外観 In フォント
プレビュー
Post by Norman Diamond
it says Monospace Oh now I see, next
  to フォントサイズ where it lets me change the size,
there's a
Post by Norman Diamond
Post by Norman Diamond
clickable button to select a font.
  In that panel it says モノスペース instead of Monospace.
  Thank you for the workaround.  I will experiment with
some
Post by Norman Diamond
Post by Norman Diamond
fonts.  I would like to
  recommend that the preview box should include some
full-width
Post by Norman Diamond
Post by Norman Diamond
characters so that users can
  judge more easily.  If a font doesn't include
glyphs for
Post by Norman Diamond
Post by Norman Diamond
full-width characters then users
  will see immediately that they shouldn't
experiment with
Post by Norman Diamond
Post by Norman Diamond
such a font.
  I still think there's a bug.
  Konsole knows that the glyph for 字 is full-width, so
Konsole
Post by Norman Diamond
Post by Norman Diamond
should know that the glyph for
  downarrow is full-width.
  The version of Konsole that was included in Suse 11.0
did know
Post by Norman Diamond
Post by Norman Diamond
that downarrow is
  full-width.  I never had to look at what the default
font was
Post by Norman Diamond
Post by Norman Diamond
because it just worked.
  Yours sincerely, Norman Diamond
  Ok, Monospace is an alias of IPAGothic on your
environment.
Post by Norman Diamond
Post by Norman Diamond
  The font, IPAGothic provides double-width
down-allow glyph
Post by Norman Diamond
Post by Norman Diamond
although Konsole assumes that
  the glyph should have the same width as glyphs
such as
Post by Norman Diamond
Post by Norman Diamond
'a' and '0'.
  A solution is to use another font such as DejaVu
Sans Mono.
Post by Norman Diamond
Post by Norman Diamond
  Settings > Edit Current Profile... >
Appearance
Post by Norman Diamond
Post by Norman Diamond
  If you do not need Japanese fonts, please
uninstall
Post by Norman Diamond
Post by Norman Diamond
ipa-gothic-fonts. So another font is
  to be selected for Monospace.
  Fuminobu TAKEYAMA
  Yes the font is Monospace.  I did not change
it from
Post by Norman Diamond
Post by Norman Diamond
the default.  A
  Konsole settings panel displayed the name
Monospace and
Post by Norman Diamond
Post by Norman Diamond
allowed me to
  change the
  font size but didn't allow me to select a
different
Post by Norman Diamond
Post by Norman Diamond
font.
  For this e-mail I use Yahoo's web
interface from
Post by Norman Diamond
Post by Norman Diamond
Internet Explorer
  under Windows, so I translated the output of
fc-match -v
Post by Norman Diamond
Post by Norman Diamond
and fc-match -v Monospace from
  UTF-8 to Microsoft's Unicode encoding. 
Yahoo's web
Post by Norman Diamond
Post by Norman Diamond
interface will scramble the text that
  I will paste below, so I ALSO attach
  the
  outputs as files.
  Output of fc-match -v
"IPA
Post by Norman Diamond
Post by Norman Diamond
"IPAPGothic"(w) "IPA P
  ゴシック"(w) fullnamelang: "en"(w)
  True(w) hintstyle: 0(i)(w) hinting: True(w)
verticallayout: False(s) autohint: True(w)
"/usr/share/fonts/truetype/ipagp.ttf"(w) index: 0(i)(w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
Post by Norman Diamond
Post by Norman Diamond
fontversion: 198574(i)(s)
  capability: "otlayout:hani otlayout:kana
otlayout:latn"(w) fontformat: "TrueType"(w)
  embeddedbitmap: True(w) decorative: False(s)
0(i)(w) namelang: "ja"(s)
"sha256:a63f6153841e56ec9df1b31a54d98d3f41d8aaa1c3d8df30a33502296f5b8068"(w)
Post by Norman Diamond
Post by Norman Diamond
  postscriptname: "IPAPGothic"(w)
force_hintstyle: "none"(w) force_autohint: False(w)
False(w)
Post by Norman Diamond
Post by Norman Diamond
  Output of fc-match -v Monospace
"ja"(s)
"Regular"(w)
Post by Norman Diamond
Post by Norman Diamond
stylelang: "en"(w) fullname: "IPAGothic"(w)
"IPAゴシック"(w)
  fullnamelang: "en"(w)
"ja"(w)
90(i)(w)
Post by Norman Diamond
Post by Norman Diamond
  True(w) hintstyle: 0(i)(w) hinting: True(w)
verticallayout: False(s) autohint: True(w)
"/usr/share/fonts/truetype/ipag.ttf"(w) index: 0(i)(w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
Post by Norman Diamond
Post by Norman Diamond
fontversion: 198574(i)(s)
  capability: "otlayout:hani otlayout:kana
otlayout:latn"(w) fontformat: "TrueType"(w)
  embeddedbitmap: False(w) decorative: False(s)
lcdfilter: 0(i)(w) namelang: "ja"(s)
"sha256:503af4a8b84d1079b8e2e358dc7f7a7fb8cb7a1f212f35eaef6782dbfc75a55e"(w)
Post by Norman Diamond
Post by Norman Diamond
  postscriptname: "IPAGothic"(w)
force_hintstyle: "none"(w) force_autohint: False(w)
False(w)
Post by Norman Diamond
Post by Norman Diamond
  ----- Original Message -----
  From: Fuminobu TAKEYAMA
[opensuse-m17n] Konsole displays some
  characters
  wrong
  Hi,
  Which font do you use on Konsole?
  Then, can you share the result of
fc-match -v?
Post by Norman Diamond
Post by Norman Diamond
If the font is "Monospace" then run
  the following
  $ fc-match -v Monospace
  # you can snip off
"charset:" section
Post by Norman Diamond
Post by Norman Diamond
  Fuminobu TAKEYAMA
  On 2015/06/09 11:41, Norman Diamond
  In openSUSE 13.2, in Konsole,
downarrow↓displays↓wrong. The file encoding,
  Konsole terminal's encoding,
and
Post by Norman Diamond
Post by Norman Diamond
  vim's
  encoding
  are all UTF-8.
Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three. vim counts characters
properly
Post by Norman Diamond
Post by Norman Diamond
  but the display is
  wrong.
  more, cat, etc., display wrong.
  I used Suse 11.0 for six years. 
Konsole
Post by Norman Diamond
Post by Norman Diamond
had no problem
  with
  downarrows.
  Yours sincerely, Norman Diamond
--
--
--
--
To unsubscribe, e-mail: opensuse-m17n+***@opensuse.org
To contact the owner, e-mail: opensuse-m17n+***@opensuse.org
Takashi Iwai
2015-06-13 18:58:54 UTC
Permalink
At Sat, 13 Jun 2015 16:56:57 +0900 (JST),
Post by Norman Diamond
OK, it could be 20 years too late to prevent ambiguous-width Romaji, katakana, and space characters from entering Unicode.
However, the Unicode consortium continues to add characters.  It isn't too late to add some necessary characters.
A half-width downarrow can be added with the same width as half-width space, Romaji, and katakana.  A full-width downarrow can be added with same same width as full-width space, Romaji, and katakana.  Ambiguous-width space, Romaji, and katakana can be added to help unify the performance of applications when the text doesn't happen to include arrows.
You can try argue with Unicode people, but IMO it's against the
philosophy of Unicode. In the history or Unicode, such ambiguity
isn't resolved by different characters but with different attributes.

Please read articles about Unicode. There have been many discussions
about CJK, especially glyphs shared by CJK but can be rendered
differently. The full/half width is one of such arguments.

But, as I already wrote, you should try globaladvance option at
first.


Takashi
Post by Norman Diamond
Sincerely,
Norman Diamond
Post by Norman Diamond
Post by Norman Diamond
I think the Unicode consortium should be taught that full-width and
half-width downarrow can be distinguished just as full-width Romaji can be
distinguished from half-width Romaji, without even needing CJK.  Even a
full-width plain blank space is distinguished from a half-width plain blank
space.
It's 20 years too late :)
Post by Norman Diamond
However, anyone who edits these source files will likely have CJK fonts. 
The comments are mostly in Japanese.  Strings have mostly been moved to data
files, but changes to the code are generally accompanied by adding or editing
strings that are also Japanese.
There is globaladvance attribute in the fontconfig and we used to set
it false for CJK for some time.  The attribute itself is marked as
deprecated nowadays, but this might still work.  You need to give it a
try.
Takashi
Post by Norman Diamond
Yours sincerely,
Norman Diamond
Post by Norman Diamond
  I do not agree with your statement that downarrow should be
displayed as
Post by Norman Diamond
Post by Norman Diamond
half-width.  It is still
  the same character that it used to be in ShiftJIS and EUC
encodings, and
Post by Norman Diamond
Post by Norman Diamond
the adoption of Unicode
  and UTF-8 does not change the width of the glyph.
The ambiguous width problem have been discussed for a long time.
There is no way to distinguish CJK down arrow and non-CJK down arrow.
  If I the character becomes half-width when I edit source files in
Konsole,
Post by Norman Diamond
Post by Norman Diamond
but it's still
  full-width when other people edit the same source files in a
Gnome terminal
Post by Norman Diamond
Post by Norman Diamond
or even in Windows,
  it will result in people scrambling source files.
If they edit GNOME terminal without CJK fonts, arrows are half-width.
Even if we resolve this Konsole bug, your problem will not be
resolved.
Post by Norman Diamond
Post by Norman Diamond
You still need to keep in mind that down arrow (and some other
characters) have
Post by Norman Diamond
Post by Norman Diamond
ambiguous width.
  I do not know who decided that downarrow should have an ambiguous
width.
Post by Norman Diamond
Post by Norman Diamond
  If the Unicode consortium decided that, they're causing
problems.
Post by Norman Diamond
Post by Norman Diamond
http://www.unicode.org/reports/tr11/tr11-28.html#Ambiguous
http://ja.wikipedia.org/wiki/東アジアの文字幅
Anyway, let's fix this Konsole's bug.
It is quite difficult to propose a new Unicode specification that can
be
Post by Norman Diamond
Post by Norman Diamond
acceptable for both CJK and non-CJK people.
Fuminobu TAKEYAMA
  Since the font selection dialog's preview uses Qt widgets
instead of
Post by Norman Diamond
Post by Norman Diamond
whatever Konsole uses, we
  know that the full-width downarrow character is displayed as
full-width by
Post by Norman Diamond
Post by Norman Diamond
these Linux tools.
  Furthermore in Suse 11.0 Konsole itself also displayed the
full-width
Post by Norman Diamond
Post by Norman Diamond
downarrow character as
  full-width.  So I think this bug is in the newer version of
Konsole itself,
Post by Norman Diamond
Post by Norman Diamond
not in the rest of
  Linux.  I can't say if the preview window should be removed
because it
Post by Norman Diamond
Post by Norman Diamond
doesn't really provide a
  preview, but it helped narrow down this bug.
  I do not agree with your statement that downarrow should be
displayed as
Post by Norman Diamond
Post by Norman Diamond
half-width.  It is still
  the same character that it used to be in ShiftJIS and EUC
encodings, and
Post by Norman Diamond
Post by Norman Diamond
the adoption of Unicode
  and UTF-8 does not change the width of the glyph.
  If I the character becomes half-width when I edit source files in
Konsole,
Post by Norman Diamond
Post by Norman Diamond
but it's still
  full-width when other people edit the same source files in a
Gnome terminal
Post by Norman Diamond
Post by Norman Diamond
or even in Windows,
  it will result in people scrambling source files.
  Full-width a and half-width a do not have ambiguous widths. 
Full-width ア
Post by Norman Diamond
Post by Norman Diamond
and half-width ア do not
  have ambiguous widths.  I do not know who decided that downarrow
should
Post by Norman Diamond
Post by Norman Diamond
have an ambiguous width.
  If the Unicode consortium decided that, they're causing
problems.
Post by Norman Diamond
Post by Norman Diamond
  Yours sincerely, Norman Diamond
  ----- Forwarded Message -----
  01:30 Subject: Re: [opensuse-m17n] Konsole displays some
characters
Post by Norman Diamond
Post by Norman Diamond
wrong
  I still think there's a bug.
  Yes, this is clearly a Japanese specific bug. Could you file
a bug
Post by Norman Diamond
Post by Norman Diamond
report for openSUSE and
  KDE?
  I saw a patch to fix the same problem with emacs about 5
years ago. So
Post by Norman Diamond
Post by Norman Diamond
I think this problem can
  be resolved by some hacks.
  Konsole knows that the glyph for 字 is full-width, so
Konsole
Post by Norman Diamond
Post by Norman Diamond
should know that the glyph for
  downarrow is full-width.
  Down arrow is known as ambiguous-width character. Monospaced
font
Post by Norman Diamond
Post by Norman Diamond
should provide half-width
  down arrow but some Japanese fonts such as IPA Gothic and MS
Gothic do
Post by Norman Diamond
Post by Norman Diamond
full-width down arrow
  from a kind of historical reason.
  The version of Konsole that was included in Suse 11.0
did know
Post by Norman Diamond
Post by Norman Diamond
that downarrow is
  full-width.  I never had to look at what the default
font was
Post by Norman Diamond
Post by Norman Diamond
because it just worked.
  Konsole's character layouting was changed recently as far
as I know
Post by Norman Diamond
Post by Norman Diamond
so that some
  (flexible-width) bold fonts can be placed correctly.
  I guess it causes this problem.
  In the 固定幅フォントを選択 - Konsole panel, the bottom part is a
text box. I
Post by Norman Diamond
Post by Norman Diamond
can type or paste text in
  the text box.
I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they
Post by Norman Diamond
Post by Norman Diamond
display correctly in that
  text box. Konsole corrupts the display in its main
window, and the
Post by Norman Diamond
Post by Norman Diamond
preview window does not
  preview the
  corruption.
  Unfortunately, the preview is for font selection dialog and
Qt widgets.
Post by Norman Diamond
Post by Norman Diamond
It is not preview of
  Konsole.
  Maybe, the preview should be removed.
  Fuminobu TAKEYAMA
  There is a worse bug.
  In the 固定幅フォントを選択 - Konsole panel, the bottom part is a
text box. I
Post by Norman Diamond
Post by Norman Diamond
can type or paste text in
  the text box.
I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they
Post by Norman Diamond
Post by Norman Diamond
display correctly in that
  text box. Konsole corrupts the display in its main
window, and the
Post by Norman Diamond
Post by Norman Diamond
preview window does not
  preview the corruption.
  Yours sincerely,Norman Diamond
  I click 設定 > 現在のプロファイルを編集(E)... > 外観 In フォント
プレビュー
Post by Norman Diamond
it says Monospace Oh now I see, next
  to フォントサイズ where it lets me change the size,
there's a
Post by Norman Diamond
Post by Norman Diamond
clickable button to select a font.
  In that panel it says モノスペース instead of Monospace.
  Thank you for the workaround.  I will experiment with
some
Post by Norman Diamond
Post by Norman Diamond
fonts.  I would like to
  recommend that the preview box should include some
full-width
Post by Norman Diamond
Post by Norman Diamond
characters so that users can
  judge more easily.  If a font doesn't include
glyphs for
Post by Norman Diamond
Post by Norman Diamond
full-width characters then users
  will see immediately that they shouldn't
experiment with
Post by Norman Diamond
Post by Norman Diamond
such a font.
  I still think there's a bug.
  Konsole knows that the glyph for 字 is full-width, so
Konsole
Post by Norman Diamond
Post by Norman Diamond
should know that the glyph for
  downarrow is full-width.
  The version of Konsole that was included in Suse 11.0
did know
Post by Norman Diamond
Post by Norman Diamond
that downarrow is
  full-width.  I never had to look at what the default
font was
Post by Norman Diamond
Post by Norman Diamond
because it just worked.
  Yours sincerely, Norman Diamond
  Ok, Monospace is an alias of IPAGothic on your
environment.
Post by Norman Diamond
Post by Norman Diamond
  The font, IPAGothic provides double-width
down-allow glyph
Post by Norman Diamond
Post by Norman Diamond
although Konsole assumes that
  the glyph should have the same width as glyphs
such as
Post by Norman Diamond
Post by Norman Diamond
'a' and '0'.
  A solution is to use another font such as DejaVu
Sans Mono.
Post by Norman Diamond
Post by Norman Diamond
  Settings > Edit Current Profile... >
Appearance
Post by Norman Diamond
Post by Norman Diamond
  If you do not need Japanese fonts, please
uninstall
Post by Norman Diamond
Post by Norman Diamond
ipa-gothic-fonts. So another font is
  to be selected for Monospace.
  Fuminobu TAKEYAMA
  Yes the font is Monospace.  I did not change
it from
Post by Norman Diamond
Post by Norman Diamond
the default.  A
  Konsole settings panel displayed the name
Monospace and
Post by Norman Diamond
Post by Norman Diamond
allowed me to
  change the
  font size but didn't allow me to select a
different
Post by Norman Diamond
Post by Norman Diamond
font.
  For this e-mail I use Yahoo's web
interface from
Post by Norman Diamond
Post by Norman Diamond
Internet Explorer
  under Windows, so I translated the output of
fc-match -v
Post by Norman Diamond
Post by Norman Diamond
and fc-match -v Monospace from
  UTF-8 to Microsoft's Unicode encoding. 
Yahoo's web
Post by Norman Diamond
Post by Norman Diamond
interface will scramble the text that
  I will paste below, so I ALSO attach
  the
  outputs as files.
  Output of fc-match -v
"IPA
Post by Norman Diamond
Post by Norman Diamond
"IPAPGothic"(w) "IPA P
  ゴシック"(w) fullnamelang: "en"(w)
  True(w) hintstyle: 0(i)(w) hinting: True(w)
verticallayout: False(s) autohint: True(w)
"/usr/share/fonts/truetype/ipagp.ttf"(w) index: 0(i)(w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
Post by Norman Diamond
Post by Norman Diamond
fontversion: 198574(i)(s)
  capability: "otlayout:hani otlayout:kana
otlayout:latn"(w) fontformat: "TrueType"(w)
  embeddedbitmap: True(w) decorative: False(s)
0(i)(w) namelang: "ja"(s)
"sha256:a63f6153841e56ec9df1b31a54d98d3f41d8aaa1c3d8df30a33502296f5b8068"(w)
Post by Norman Diamond
Post by Norman Diamond
  postscriptname: "IPAPGothic"(w)
force_hintstyle: "none"(w) force_autohint: False(w)
False(w)
Post by Norman Diamond
Post by Norman Diamond
  Output of fc-match -v Monospace
"ja"(s)
"Regular"(w)
Post by Norman Diamond
Post by Norman Diamond
stylelang: "en"(w) fullname: "IPAGothic"(w)
"IPAゴシック"(w)
  fullnamelang: "en"(w)
"ja"(w)
90(i)(w)
Post by Norman Diamond
Post by Norman Diamond
  True(w) hintstyle: 0(i)(w) hinting: True(w)
verticallayout: False(s) autohint: True(w)
"/usr/share/fonts/truetype/ipag.ttf"(w) index: 0(i)(w)
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
Post by Norman Diamond
Post by Norman Diamond
fontversion: 198574(i)(s)
  capability: "otlayout:hani otlayout:kana
otlayout:latn"(w) fontformat: "TrueType"(w)
  embeddedbitmap: False(w) decorative: False(s)
lcdfilter: 0(i)(w) namelang: "ja"(s)
"sha256:503af4a8b84d1079b8e2e358dc7f7a7fb8cb7a1f212f35eaef6782dbfc75a55e"(w)
Post by Norman Diamond
Post by Norman Diamond
  postscriptname: "IPAGothic"(w)
force_hintstyle: "none"(w) force_autohint: False(w)
False(w)
Post by Norman Diamond
Post by Norman Diamond
  ----- Original Message -----
  From: Fuminobu TAKEYAMA
[opensuse-m17n] Konsole displays some
  characters
  wrong
  Hi,
  Which font do you use on Konsole?
  Then, can you share the result of
fc-match -v?
Post by Norman Diamond
Post by Norman Diamond
If the font is "Monospace" then run
  the following
  $ fc-match -v Monospace
  # you can snip off
"charset:" section
Post by Norman Diamond
Post by Norman Diamond
  Fuminobu TAKEYAMA
  On 2015/06/09 11:41, Norman Diamond
  In openSUSE 13.2, in Konsole,
downarrow↓displays↓wrong. The file encoding,
  Konsole terminal's encoding,
and
Post by Norman Diamond
Post by Norman Diamond
  vim's
  encoding
  are all UTF-8.
Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three. vim counts characters
properly
Post by Norman Diamond
Post by Norman Diamond
  but the display is
  wrong.
  more, cat, etc., display wrong.
  I used Suse 11.0 for six years. 
Konsole
Post by Norman Diamond
Post by Norman Diamond
had no problem
  with
  downarrows.
  Yours sincerely, Norman Diamond
--
--
--
--
--
To unsubscribe, e-mail: opensuse-m17n+***@opensuse.org
To contact the owner, e-mail: opensuse-m17n+***@opensuse.org
Norman Diamond
2015-06-13 21:31:11 UTC
Permalink
I understand that the Unicode consortium had many fights, for example on unifying the Japanese and Chinese glyphs for 骨 while inventing redundant glyphs for π and π。 Except now I fail in this message, because JIS copies of Gurishaji are full-width while the Greek originals are half-width.

I don't have time to read all their arguments.  I take your word for it (and Takeyama san's word for it) that the Unicode consortium really did more stupid things.

Somehow the Unicode consortium considered it NOT against the philosophy of Unicode to copy both half-width and full-width blank space, Romaji, and katakana with separate encodings.  Where mathematicians adopted the Greek letter pi as the Greek letter pi with a new meaning, the Unicode consortium considered it NOT against their philosophy to invent two identical same-width glyphs with attributes that mathematicians considered identical (Greek letter pi) but which Unicoders considered different attributes, and Unicoders made separate encodings for the two.

So I think they can be taught that full-width downarrow can be copied from some languages into Unicode, half-width downarrow can be copied from some languages into Unicode, and ambiguous-width Romaji and Gurishaji can be invented, to improve the consistency of Unicode.

I might take a look at the globaladvance option at second, but it's too late to be first.  First I tried Takeyama san's suggestion selecting a different font in Konsole's settings, and it worked.  It's good enough for editing source files.

Sincerely,Norman Diamond
Post by Norman Diamond
Post by Norman Diamond
OK, it could be 20 years too late to prevent ambiguous-width Romaji,
katakana, and space characters from entering Unicode.
Post by Norman Diamond
However, the Unicode consortium continues to add characters.  It isn't
too late to add some necessary characters.
Post by Norman Diamond
A half-width downarrow can be added with the same width as half-width
space, Romaji, and katakana.  A full-width downarrow can be added with same same
width as full-width space, Romaji, and katakana.  Ambiguous-width space, Romaji,
and katakana can be added to help unify the performance of applications when the
text doesn't happen to include arrows.
You can try argue with Unicode people, but IMO it's against the
philosophy of Unicode.  In the history or Unicode, such ambiguity
isn't resolved by different characters but with different attributes.
Please read articles about Unicode.  There have been many discussions
about CJK, especially glyphs shared by CJK but can be rendered
differently.  The full/half width is one of such arguments.
But, as I already wrote, you should try globaladvance option at
first.
Takashi
Post by Norman Diamond
Sincerely,
Norman Diamond
Post by Norman Diamond
  I think the Unicode consortium should be taught that full-width
and
Post by Norman Diamond
Post by Norman Diamond
half-width downarrow can be distinguished just as full-width Romaji
can be
Post by Norman Diamond
Post by Norman Diamond
distinguished from half-width Romaji, without even needing CJK.  Even
a
Post by Norman Diamond
Post by Norman Diamond
full-width plain blank space is distinguished from a half-width plain
blank
Post by Norman Diamond
Post by Norman Diamond
space.
It's 20 years too late :)
  However, anyone who edits these source files will likely have CJK
fonts. 
Post by Norman Diamond
Post by Norman Diamond
The comments are mostly in Japanese.  Strings have mostly been moved
to data
Post by Norman Diamond
Post by Norman Diamond
files, but changes to the code are generally accompanied by adding or
editing
Post by Norman Diamond
Post by Norman Diamond
strings that are also Japanese.
There is globaladvance attribute in the fontconfig and we used to set
it false for CJK for some time.  The attribute itself is marked as
deprecated nowadays, but this might still work.  You need to give it a
try.
Takashi
  Yours sincerely,
  Norman Diamond
  >>  I do not agree with your statement that downarrow
should be
Post by Norman Diamond
Post by Norman Diamond
displayed as
  > half-width.  It is still
  >>  the same character that it used to be in ShiftJIS and
EUC
Post by Norman Diamond
Post by Norman Diamond
encodings, and
  > the adoption of Unicode
  >>  and UTF-8 does not change the width of the glyph.
  >
  > The ambiguous width problem have been discussed for a long
time.
Post by Norman Diamond
Post by Norman Diamond
  > There is no way to distinguish CJK down arrow and non-CJK
down arrow.
Post by Norman Diamond
Post by Norman Diamond
  >
  >
  >>  If I the character becomes half-width when I edit
source files in
Post by Norman Diamond
Post by Norman Diamond
Konsole,
  > but it's still
  >>  full-width when other people edit the same source files
in a
Post by Norman Diamond
Post by Norman Diamond
Gnome terminal
  > or even in Windows,
  >>  it will result in people scrambling source files.
  >
  > If they edit GNOME terminal without CJK fonts, arrows are
half-width.
Post by Norman Diamond
Post by Norman Diamond
  > Even if we resolve this Konsole bug, your problem will not
be
Post by Norman Diamond
Post by Norman Diamond
resolved.
  >
  > You still need to keep in mind that down arrow (and some
other
Post by Norman Diamond
Post by Norman Diamond
characters) have
  > ambiguous width.
  >
  >
  >
  >>  I do not know who decided that downarrow should have an
ambiguous
Post by Norman Diamond
Post by Norman Diamond
width.
  >>  If the Unicode consortium decided that, they're
causing
Post by Norman Diamond
Post by Norman Diamond
problems.
  >
  > http://www.unicode.org/reports/tr11/tr11-28.html#Ambiguous
  > http://ja.wikipedia.org/wiki/東アジアの文字幅
  >
  >
  > Anyway, let's fix this Konsole's bug.
  >
  > It is quite difficult to propose a new Unicode specification
that can
Post by Norman Diamond
Post by Norman Diamond
be
  > acceptable for both CJK and non-CJK people.
  >
  > Fuminobu TAKEYAMA
  >
  >>  Since the font selection dialog's preview uses Qt
widgets
Post by Norman Diamond
Post by Norman Diamond
instead of
  > whatever Konsole uses, we
  >>  know that the full-width downarrow character is
displayed as
Post by Norman Diamond
Post by Norman Diamond
full-width by
  > these Linux tools.
  >>  Furthermore in Suse 11.0 Konsole itself also displayed
the
Post by Norman Diamond
Post by Norman Diamond
full-width
  > downarrow character as
  >>  full-width.  So I think this bug is in the newer
version of
Post by Norman Diamond
Post by Norman Diamond
Konsole itself,
  > not in the rest of
  >>  Linux.  I can't say if the preview window should be
removed
Post by Norman Diamond
Post by Norman Diamond
because it
  > doesn't really provide a
  >>  preview, but it helped narrow down this bug.
  >>
  >>  I do not agree with your statement that downarrow
should be
Post by Norman Diamond
Post by Norman Diamond
displayed as
  > half-width.  It is still
  >>  the same character that it used to be in ShiftJIS and
EUC
Post by Norman Diamond
Post by Norman Diamond
encodings, and
  > the adoption of Unicode
  >>  and UTF-8 does not change the width of the glyph.
  >>
  >>  If I the character becomes half-width when I edit
source files in
Post by Norman Diamond
Post by Norman Diamond
Konsole,
  > but it's still
  >>  full-width when other people edit the same source files
in a
Post by Norman Diamond
Post by Norman Diamond
Gnome terminal
  > or even in Windows,
  >>  it will result in people scrambling source files.
  >>
  >>  Full-width a and half-width a do not have ambiguous
widths. 
Post by Norman Diamond
Post by Norman Diamond
Full-width ア
  > and half-width ア do not
  >>  have ambiguous widths.  I do not know who decided that
downarrow
Post by Norman Diamond
Post by Norman Diamond
should
  > have an ambiguous width.
  >>  If the Unicode consortium decided that, they're
causing
Post by Norman Diamond
Post by Norman Diamond
problems.
  >>
  >>  Yours sincerely, Norman Diamond
  >>
  >>
  >>  ----- Forwarded Message -----
  >>>  01:30 Subject: Re: [opensuse-m17n] Konsole displays
some
Post by Norman Diamond
Post by Norman Diamond
characters
  > wrong
  >>>
  >>>>>  I still think there's a bug.
  >>>
  >>>  Yes, this is clearly a Japanese specific bug. Could
you file
Post by Norman Diamond
Post by Norman Diamond
a bug
  > report for openSUSE and
  >>>  KDE?
  >>>
  >>>  I saw a patch to fix the same problem with emacs
about 5
Post by Norman Diamond
Post by Norman Diamond
years ago. So
  > I think this problem can
  >>>  be resolved by some hacks.
  >>>
  >>>
  >>>>>  Konsole knows that the glyph for 字 is
full-width, so
Post by Norman Diamond
Post by Norman Diamond
Konsole
  > should know that the glyph for
  >>>>>  downarrow is full-width.
  >>>
  >>>  Down arrow is known as ambiguous-width character.
Monospaced
Post by Norman Diamond
Post by Norman Diamond
font
  > should provide half-width
  >>>  down arrow but some Japanese fonts such as IPA
Gothic and MS
Post by Norman Diamond
Post by Norman Diamond
Gothic do
  > full-width down arrow
  >>>  from a kind of historical reason.
  >>>
  >>>
  >>>>>  The version of Konsole that was included in
Suse 11.0
Post by Norman Diamond
Post by Norman Diamond
did know
  > that downarrow is
  >>>>>  full-width.  I never had to look at what
the default
Post by Norman Diamond
Post by Norman Diamond
font was
  > because it just worked.
  >>>
  >>>  Konsole's character layouting was changed
recently as far
Post by Norman Diamond
Post by Norman Diamond
as I know
  > so that some
  >>>  (flexible-width) bold fonts can be placed
correctly.
Post by Norman Diamond
Post by Norman Diamond
  >>>
  >>>  I guess it causes this problem.
  >>>
  >>>
  >>>>  In the 固定幅フォントを選択 - Konsole panel, the bottom
part is a
Post by Norman Diamond
Post by Norman Diamond
text box. I
  > can type or paste text in
  >>>>  the text box.
I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they
  > display correctly in that
  >>>>  text box. Konsole corrupts the display in its
main
Post by Norman Diamond
Post by Norman Diamond
window, and the
  > preview window does not
  >>>>  preview the
  >>>  corruption.
  >>>
  >>>  Unfortunately, the preview is for font selection
dialog and
Post by Norman Diamond
Post by Norman Diamond
Qt widgets.
  > It is not preview of
  >>>  Konsole.
  >>>
  >>>  Maybe, the preview should be removed.
  >>>
  >>>
  >>>  Fuminobu TAKEYAMA
  >>>
  >>>>  There is a worse bug.
  >>>>
  >>>>  In the 固定幅フォントを選択 - Konsole panel, the bottom
part is a
Post by Norman Diamond
Post by Norman Diamond
text box. I
  > can type or paste text in
  >>>>  the text box.
I↓↓↓↓↓can↓↓↓↓↓paste↓↓↓↓↓five↓↓↓↓↓downarrows, and they
  > display correctly in that
  >>>>  text box. Konsole corrupts the display in its
main
Post by Norman Diamond
Post by Norman Diamond
window, and the
  > preview window does not
  >>>>  preview the corruption.
  >>>>
  >>>>  Yours sincerely,Norman Diamond
  >>>>
  >>>>
  >>>>
  >>>>>  I click 設定 > 現在のプロファイルを編集(E)... > 外観
In フォント
Post by Norman Diamond
Post by Norman Diamond
  プレビュー
  > it says Monospace Oh now I see, next
  >>>>>  to フォントサイズ where it lets me change the
size,
Post by Norman Diamond
Post by Norman Diamond
there's a
  > clickable button to select a font.
  >>>>>  In that panel it says モノスペース instead of
Monospace.
Post by Norman Diamond
Post by Norman Diamond
  >>>>>
  >>>>>  Thank you for the workaround.  I will
experiment with
Post by Norman Diamond
Post by Norman Diamond
some
  > fonts.  I would like to
  >>>>>  recommend that the preview box should
include some
Post by Norman Diamond
Post by Norman Diamond
full-width
  > characters so that users can
  >>>>>  judge more easily.  If a font doesn't
include
Post by Norman Diamond
Post by Norman Diamond
glyphs for
  > full-width characters then users
  >>>>>  will see immediately that they
shouldn't
Post by Norman Diamond
Post by Norman Diamond
experiment with
  > such a font.
  >>>>>
  >>>>>  I still think there's a bug.
  >>>>>
  >>>>>  Konsole knows that the glyph for 字 is
full-width, so
Post by Norman Diamond
Post by Norman Diamond
Konsole
  > should know that the glyph for
  >>>>>  downarrow is full-width.
  >>>>>
  >>>>>  The version of Konsole that was included in
Suse 11.0
Post by Norman Diamond
Post by Norman Diamond
did know
  > that downarrow is
  >>>>>  full-width.  I never had to look at what
the default
Post by Norman Diamond
Post by Norman Diamond
font was
  > because it just worked.
  >>>>>
  >>>>>  Yours sincerely, Norman Diamond
  >>>>>
  >>>>>
  >>>>>
  >>>>>>  Ok, Monospace is an alias of IPAGothic
on your
Post by Norman Diamond
Post by Norman Diamond
environment.
  >>>>>>
  >>>>>>  The font, IPAGothic provides
double-width
Post by Norman Diamond
Post by Norman Diamond
down-allow glyph
  > although Konsole assumes that
  >>>>>>  the glyph should have the same width as
glyphs
Post by Norman Diamond
Post by Norman Diamond
such as
  > 'a' and '0'.
  >>>>>>
  >>>>>>  A solution is to use another font such
as DejaVu
Post by Norman Diamond
Post by Norman Diamond
Sans Mono.
  >>>>>>  Settings > Edit Current Profile...
Appearance
  >>>>>>
  >>>>>>  If you do not need Japanese fonts,
please
Post by Norman Diamond
Post by Norman Diamond
uninstall
  > ipa-gothic-fonts. So another font is
  >>>>>>  to be selected for Monospace.
  >>>>>>
  >>>>>>
  >>>>>>  Fuminobu TAKEYAMA
  >>>>>>
  >>>>>>
  >>>>>>  On 2015/06/11 6:42, Norman Diamond
  >>>>>>>  Yes the font is Monospace.  I did
not change
Post by Norman Diamond
Post by Norman Diamond
it from
  > the default.  A
  >>>>>>  Konsole settings panel displayed the
name
Post by Norman Diamond
Post by Norman Diamond
Monospace and
  > allowed me to
  >>>>>  change the
  >>>>>>  font size but didn't allow me to
select a
Post by Norman Diamond
Post by Norman Diamond
different
  > font.
  >>>>>>>
  >>>>>>>  For this e-mail I use Yahoo's
web
Post by Norman Diamond
Post by Norman Diamond
interface from
  > Internet Explorer
  >>>>>
  >>>>>>  under Windows, so I translated the
output of
Post by Norman Diamond
Post by Norman Diamond
fc-match -v
  > and fc-match -v Monospace from
  >>>>>>  UTF-8 to Microsoft's Unicode
encoding. 
Post by Norman Diamond
Post by Norman Diamond
Yahoo's web
  > interface will scramble the text that
  >>>>>>  I will paste below, so I ALSO attach
  >>>>>  the
  >>>>>>  outputs as files.
  >>>>>>>
  >>>>>>>
  >>>>>>>  Output of fc-match -v
  >>>>>>>
  >>>>>>>  Pattern has 41 elts (size 48)
"IPA
  >>>>>>>  "ja"(s) "en"(s)
  > "Regular"(w) stylelang: "en"(w)
  > "IPAPGothic"(w) "IPA P
"en"(w)
True(w)
Post by Norman Diamond
Post by Norman Diamond
  > verticallayout: False(s) autohint: True(w)
0(i)(w)
Post by Norman Diamond
Post by Norman Diamond
  >>>>>>>  outline: True(w) scalable: True(w)
  >>>>>>
  >>>>>
  >
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
Post by Norman Diamond
Post by Norman Diamond
  >>>>>>>
  >>>>>
  > fontversion: 198574(i)(s)
  >>>>>>>  capability: "otlayout:hani
otlayout:kana
Post by Norman Diamond
Post by Norman Diamond
  > otlayout:latn"(w) fontformat: "TrueType"(w)
False(s)
Post by Norman Diamond
Post by Norman Diamond
  > 0(i)(w) namelang: "ja"(s)
  >>>>>>>  prgname: "fc-match"(s)
  >>>>>>
  >>>>>
  >
"sha256:a63f6153841e56ec9df1b31a54d98d3f41d8aaa1c3d8df30a33502296f5b8068"(w)
"IPAPGothic"(w)
False(w)
Post by Norman Diamond
Post by Norman Diamond
  >>>>>>>  force_bw: False(w)
False(w)
  >>>>>>>
  >>>>>>>  Output of fc-match -v Monospace
  >>>>>>>
  >>>>>>>  Pattern has 42 elts (size 48)
"ja"(s)
"Regular"(w)
"IPAGothic"(w)
Post by Norman Diamond
Post by Norman Diamond
  > "IPAゴシック"(w)
  >>>>>>>  fullnamelang: "en"(w)
"ja"(w)
  >>>>>>>  12(f)(s) pixelsize: 12.5(f)(s)
90(i)(w)
True(w)
Post by Norman Diamond
Post by Norman Diamond
  > verticallayout: False(s) autohint: True(w)
0(i)(w)
Post by Norman Diamond
Post by Norman Diamond
  >>>>>>>  outline: True(w) scalable: True(w)
  >>>>>>
  >>>>>
  >
aa|ay|bg|bi|br|ch|cs|da|de|en|eo|es|et|eu|fi|fj|fo|fur|fy|gd|gl|gv|ho|hu|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|nr|nso|oc|om|os|pl|pt|rm|ru|sel|sk|sma|smj|so|sq|ss|st|sv|sw|tk|tl|tn|ts|uz|vo|vot|wa|wen|xh|yap|zu|an|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|li|ms|ng|pap-an|pap-aw|qu|quz|rn|rw|sc|sg|sn|su|za(s)
Post by Norman Diamond
Post by Norman Diamond
  >>>>>>>
  >>>>>
  > fontversion: 198574(i)(s)
  >>>>>>>  capability: "otlayout:hani
otlayout:kana
Post by Norman Diamond
Post by Norman Diamond
  > otlayout:latn"(w) fontformat: "TrueType"(w)
  >>>>>>>  embeddedbitmap: False(w)
decorative: False(s)
Post by Norman Diamond
Post by Norman Diamond
  > lcdfilter: 0(i)(w) namelang: "ja"(s)
  >>>>>>>  prgname: "fc-match"(s)
  >>>>>>
  >>>>>
  >
"sha256:503af4a8b84d1079b8e2e358dc7f7a7fb8cb7a1f212f35eaef6782dbfc75a55e"(w)
"IPAGothic"(w)
False(w)
Post by Norman Diamond
Post by Norman Diamond
  >>>>>>>  force_bw: False(w)
False(w)
  >>>>>>>
  >>>>>>>
  >>>>>>>
  >>>>>>>
  >>>>>>>>
  >>>>>>>>  ----- Original Message -----
  >>>>>>>>>  From: Fuminobu TAKEYAMA
  >>>>>>>>>  2015/6/10, Wed 19:43
  > [opensuse-m17n] Konsole displays some
  >>>>>  characters
  >>>>>>  wrong
  >>>>>>>>>
  >>>>>>>>>  Hi,
  >>>>>>>>>
  >>>>>>>>>  Which font do you use on
Konsole?
Post by Norman Diamond
Post by Norman Diamond
  >>>>>>>>>
  >>>>>>>>>  Then, can you share the
result of
Post by Norman Diamond
Post by Norman Diamond
fc-match -v?
  > If the font is "Monospace" then run
  >>>>>>>>>  the following
  >>>>>>>>>  $ fc-match -v Monospace
  >>>>>>>>>
  >>>>>>>>>  # you can snip off
"charset:" section
  >>>>>>>>>
  >>>>>>>>>
  >>>>>>>>>  Fuminobu TAKEYAMA
  >>>>>>>>>
  >>>>>>>>>  On 2015/06/09 11:41, Norman
Diamond
Post by Norman Diamond
Post by Norman Diamond
  >>>>>>>>>>  In openSUSE 13.2, in
Konsole,
Post by Norman Diamond
Post by Norman Diamond
  > downarrow↓displays↓wrong. The file encoding,
  >>>>>>>>>>  Konsole terminal's
encoding,
Post by Norman Diamond
Post by Norman Diamond
and
  >>>>>
  >>>>>>  vim's
  >>>>>>>>  encoding
  >>>>>>>>>  are all UTF-8.
  >>>>>>>>>>
  > Five↓↓↓↓↓downarrows↓↓↓↓↓look↓↓↓↓↓like↓↓↓↓↓three. vim counts
characters
Post by Norman Diamond
Post by Norman Diamond
properly
  >>>>>>>>>>  but the display is
  >>>>>  wrong.
  >>>>>>>>>>  more, cat, etc.,
display wrong.
Post by Norman Diamond
Post by Norman Diamond
  >>>>>>>>>>
  >>>>>>>>>>  I used Suse 11.0 for
six years. 
Post by Norman Diamond
Post by Norman Diamond
Konsole
  > had no problem
  >>>>>  with
  >>>>>>>>  downarrows.
  >>>>>>>>>>
  >>>>>>>>>>  Yours sincerely, Norman
Diamond
Post by Norman Diamond
Post by Norman Diamond
  >>>>>>>>>>
  >>>>>>>>>  To contact the owner,
  >>>>>>>>>
  >>>>>>>>
  >>>>>>
  >>>>>
  >>>
  >>>
  >>>
  >>>
  > --
  >
  --
--
--
--
       
--
To unsubscribe, e-mail: opensuse-m17n+***@opensuse.org
To contact the owner, e-mail: opensuse-m17n+***@opensuse.org
Loading...